Oct 30 18:45:53.880878 ip-10-0-139-33 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:45:54.365394 ip-10-0-139-33 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:45:54.365394 ip-10-0-139-33 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:45:54.365394 ip-10-0-139-33 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:45:54.365394 ip-10-0-139-33 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:45:54.370923 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370853    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:54.370923 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370921    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:54.370923 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370925    2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:54.370923 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370928    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:54.370923 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370931    2570 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370934    2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370937    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370940    2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370943    2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370945    2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370948    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370950    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370953    2570 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370955    2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370958    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370961    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370963    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370966    2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370968    2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370971    2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370974    2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370977    2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370979    2570 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:54.371112 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370982    2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370986    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370988    2570 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370991    2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370994    2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370997    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.370999    2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371002    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371004    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371007    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371010    2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371012    2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371015    2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371018    2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371022    2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371027    2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371030    2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371033    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371036    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371039    2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:54.371555 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371042    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371044    2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371047    2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371049    2570 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371052    2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371055    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371059    2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371063    2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371066    2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371069    2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371072    2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371074    2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371076    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371079    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371082    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371084    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371087    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371089    2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371092    2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:54.372030 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371094    2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371097    2570 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371100    2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371102    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371105    2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371107    2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371110    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371113    2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371117    2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371120    2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371123    2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371126    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371129    2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371131    2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371134    2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371136    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371139    2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371146    2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371149    2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:54.372505 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371152    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371175    2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371181    2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371184    2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371187    2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371191    2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371193    2570 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371196    2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371199    2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371201    2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371204    2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371206    2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371209    2570 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371213    2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371218    2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371220    2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371223    2570 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371225    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371228    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371231    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:54.372965 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371233    2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371236    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371239    2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371242    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.371244    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373117    2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373126    2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373130    2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373135    2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373138    2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373140    2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373143    2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373146    2570 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373149    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373152    2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373155    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373158    2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373160    2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373163    2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:54.373444 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373165    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373168    2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373171    2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373173    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373176    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373179    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373181    2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373184    2570 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373186    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373189    2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373192    2570 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373194    2570 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373197    2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373199    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373202    2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373207    2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373210    2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373213    2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373215    2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373218    2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:54.373927 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373220    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373223    2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373225    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373228    2570 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373230    2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373233    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373236    2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373238    2570 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373241    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373244    2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373247    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373249    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373252    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373254    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373257    2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373259    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373262    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373264    2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373267    2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:54.374422 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373269    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373272    2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373274    2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373277    2570 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373279    2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373281    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373284    2570 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373286    2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373291    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373294    2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373296    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373299    2570 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373301    2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373304    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373306    2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373309    2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373311    2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373314    2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373316    2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373319    2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:54.374929 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373322    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373326    2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373330    2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373334    2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373337    2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373340    2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373342    2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373345    2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373348    2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373350    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373353    2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373355    2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373358    2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373367    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373370    2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373373    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373375    2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373378    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373380    2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:54.375403 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373383    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373387    2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373390    2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373393    2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373395    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373398    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373401    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373403    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373406    2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373410    2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373412    2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373415    2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373417    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373419    2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373926    2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373932    2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373935    2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373939    2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373942    2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373945    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:54.375862 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373948    2570 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373951    2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373953    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373956    2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373958    2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373961    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373964    2570 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373967    2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373969    2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373972    2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373974    2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373977    2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373979    2570 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373983    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373986    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373988    2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373991    2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373993    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373996    2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.373998    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:54.376342 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374000    2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374004    2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374009    2570 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374012    2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374015    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374017    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374020    2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374023    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374025    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374028    2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374031    2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374033    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374036    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374038    2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374041    2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374043    2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374046    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374049    2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374051    2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:54.376824 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374054    2570 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374057    2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374059    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374062    2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374065    2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374067    2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374070    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374088    2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374092    2570 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374095    2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374098    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374101    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374105    2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374109    2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374113    2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374116    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374119    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374121    2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374124    2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:54.377262 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374127    2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374129    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374132    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374134    2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374137    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374140    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374142    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374145    2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374148    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374150    2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374152    2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374155    2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374158    2570 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374160    2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374163    2570 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374165    2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374168    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374171    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374173    2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374176    2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:54.377737 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374180    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374182    2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374185    2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374187    2570 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374190    2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374192    2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374194    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374197    2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374199    2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374202    2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374204    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374207    2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374209    2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374212    2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374214    2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374217    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374219    2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374222    2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374225    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:54.378202 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374227    2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374230    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374232    2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374331    2570 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374339    2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374345    2570 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374351    2570 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374355    2570 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374358    2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374365    2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374377    2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374380    2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374383    2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374387    2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374390    2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374393    2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374396    2570 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374399    2570 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374402    2570 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374405    2570 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374408    2570 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374411    2570 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374414    2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:45:54.378693 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374417    2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374420    2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374424    2570 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374428    2570 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374431    2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374434    2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374438    2570 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374441    2570 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374447    2570 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374450    2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374453    2570 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374456    2570 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374461    2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374466    2570 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374469    2570 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374472    2570 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374474    2570 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374478    2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374482    2570 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374485    2570 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374488    2570 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374494    2570 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374497    2570 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374501    2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:45:54.379373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374504    2570 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374507    2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374510    2570 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374514    2570 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374517    2570 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374519    2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374522    2570 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374525    2570 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374528    2570 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374531    2570 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374535    2570 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374538    2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374541    2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374544    2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374547    2570 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374550    2570 flags.go:64] FLAG: --help="false"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374553    2570 flags.go:64] FLAG: --hostname-override="ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374557    2570 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374560    2570 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374563    2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374567    2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374571    2570 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374574    2570 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:45:54.380009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374589    2570 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374593    2570 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374596    2570 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374599    2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374602    2570 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374605    2570 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374609    2570 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374612    2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374615    2570 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374618    2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374620    2570 flags.go:64] FLAG: --lock-file=""
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374623    2570 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374627    2570 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374630    2570 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374635    2570 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374638    2570 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374641    2570 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374644    2570 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374647    2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374651    2570 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374653    2570 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374656    2570 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374661    2570 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374664    2570 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374668    2570 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:45:54.380631 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374671    2570 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374674    2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374677    2570 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374680    2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374683    2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374688    2570 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374691    2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374698    2570 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374701    2570 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374704    2570 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374707    2570 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374710    2570 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374713    2570 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374717    2570 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374720    2570 flags.go:64] FLAG: --port="10250"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374723    2570 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374726    2570 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-070fe8a2c1d4a5301"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374729    2570 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374732    2570 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374735    2570 flags.go:64] FLAG: --register-node="true"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374738    2570 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374742    2570 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374744    2570 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374747    2570 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374750    2570 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374754    2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:45:54.381251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374757    2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374760    2570 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374763    2570 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374766    2570 flags.go:64] FLAG: --runonce="false"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374769    2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374772    2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374775    2570 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374777    2570 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374780    2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374784    2570 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374787    2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374791    2570 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374794    2570 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374797    2570 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374800    2570 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374803    2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374806    2570 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374809    2570 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374812    2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374818    2570 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374821    2570 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374824    2570 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374827    2570 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374830    2570 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374833    2570 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:45:54.381900 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374836    2570 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374839    2570 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374842    2570 flags.go:64] FLAG: --v="2"
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374846    2570 flags.go:64] FLAG: --version="false"
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374850    2570 flags.go:64] FLAG: --vmodule=""
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374854    2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.374857    2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374982    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374985    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374988    2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374991    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374994    2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374997    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.374999    2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375002    2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375005    2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375007    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375010    2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375014    2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375017    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375019    2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:54.382495 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375022    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375025    2570 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375027    2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375030    2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375032    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375036    2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375039    2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375042    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375044    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375047    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375049    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375052    2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375054    2570 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375057    2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375060    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375062    2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375065    2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375067    2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375070    2570 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:54.383011 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375073    2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375075    2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375078    2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375080    2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375083    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375085    2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375088    2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375091    2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375093    2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375096    2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375100    2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375103    2570 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375106    2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375110    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375112    2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375115    2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375118    2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375120    2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375124    2570 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375127    2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:54.383464 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375130    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375132    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375135    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375137    2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375140    2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375142    2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375145    2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375148    2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375151    2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375153    2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375156    2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375158    2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375161    2570 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375164    2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375166    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375169    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375172    2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375175    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375177    2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:54.383975 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375180    2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375182    2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375185    2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375189    2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375192    2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375194    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375197    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375199    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375202    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375204    2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375207    2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375210    2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375213    2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375216    2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375218    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375221    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375224    2570 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375226    2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375229    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:54.384426 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375232    2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375234    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375237    2570 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375239    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375242    2570 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375245    2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375247    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375250    2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375252    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375257    2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375260    2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375263    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375266    2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375269    2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375271    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:54.384897 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.375296    2570 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375331    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375336    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375340    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375343    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375346    2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375349    2570 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375352    2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375356    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375359    2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375363    2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375365    2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375368    2570 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375371    2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375373    2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375376    2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375379    2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375381    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375384    2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:54.385248 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375386    2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375389    2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375392    2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375395    2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375397    2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375400    2570 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375404    2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375408    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375412    2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375415    2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375418    2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375421    2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375424    2570 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375427    2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375431    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375433    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375436    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375443    2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375446    2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375448    2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:54.385715 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375451    2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375453    2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375456    2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375460    2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375462    2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375465    2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375468    2570 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375471    2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375473    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375476    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375478    2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375481    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375483    2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375486    2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375489    2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375491    2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375494    2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375496    2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375499    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375502    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:54.386172 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375504    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375506    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375509    2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375512    2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375514    2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375517    2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375520    2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375523    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375525    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375528    2570 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375531    2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375534    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375536    2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375539    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375541    2570 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375545    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375548    2570 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375551    2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375553    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375556    2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:54.386679 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375558    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375563    2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375567    2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375570    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375573    2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375586    2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375589    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375592    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375594    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375597    2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375600    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375602    2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375605    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375608    2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375610    2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375617    2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375619    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375622    2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375626    2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:54.387147 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375628    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375631    2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375634    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375636    2570 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375640    2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375643    2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375645    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375648    2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.375652    2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.384288    2570 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.384303    2570 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.385458    2570 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.385465    2570 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385523    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385529    2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385532    2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385535    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385537    2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385540    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:54.387607 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385543    2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385546    2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385550    2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385555    2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385558    2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385561    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385564    2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385567    2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385570    2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385573    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385589    2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385592    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385595    2570 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385598    2570 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385600    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385603    2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385606    2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385608    2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385611    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:54.388059 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385613    2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385616    2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385619    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385621    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385625    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385629    2570 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385631    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385634    2570 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385637    2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385639    2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385642    2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385644    2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385647    2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385650    2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385652    2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385655    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385657    2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385660    2570 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385663    2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385665    2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:54.388523 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385668    2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385670    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385673    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385676    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385678    2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385681    2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385684    2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385686    2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385689    2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385691    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385693    2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385696    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385699    2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385702    2570 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385705    2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385707    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385710    2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385713    2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385716    2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:54.389008 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385719    2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385722    2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385724    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385727    2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385730    2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385732    2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385735    2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385738    2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385740    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385743    2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385745    2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385748    2570 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385751    2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385753    2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385756    2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385759    2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385761    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385764    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385766    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385769    2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:54.389504 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385771    2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385774    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385776    2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385780    2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385784    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385787    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385790    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385793    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385796    2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385799    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385801    2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385804    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385807    2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385810    2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385812    2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385815    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385817    2570 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385820    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385822    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:54.389990 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385825    2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385828    2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385830    2570 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.385858    2570 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385988    2570 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385992    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385995    2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.385998    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386000    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386003    2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386006    2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386009    2570 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386013    2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386016    2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386020    2570 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386023    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:54.390437 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386025    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386028    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386031    2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386034    2570 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386036    2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386039    2570 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386041    2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386044    2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386047    2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386050    2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386052    2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386055    2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386057    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386060    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386062    2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386065    2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386067    2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386070    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386072    2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:54.390828 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386075    2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386077    2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386080    2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386082    2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386084    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386087    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386089    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386092    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386094    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386097    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386099    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386102    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386104    2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386107    2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386110    2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386114    2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386117    2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386120    2570 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386122    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:54.391286 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386125    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386128    2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386130    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386133    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386136    2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386139    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386141    2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386144    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386146    2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386148    2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386151    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386153    2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386156    2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386158    2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386161    2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386164    2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386166    2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386169    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386171    2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:54.391741 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386174    2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386177    2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386180    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386182    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386185    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386188    2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386190    2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386193    2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386195    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386198    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386200    2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386203    2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386206    2570 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386209    2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386212    2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386214    2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386217    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386219    2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386222    2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:54.392187 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386224    2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386227    2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386230    2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386232    2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386235    2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386237    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386240    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386242    2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386245    2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386247    2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386250    2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386252    2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386254    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386257    2570 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386259    2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386262    2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386264    2570 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:54.392764 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386267    2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.386290    2570 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386322    2570 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386326    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386329    2570 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386331    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386334    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386337    2570 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386339    2570 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386342    2570 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386346    2570 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386350    2570 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386353    2570 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386355    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386358    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386361    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:45:54.393170 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386363    2570 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386366    2570 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386368    2570 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386371    2570 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386373    2570 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386376    2570 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386379    2570 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386381    2570 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386384    2570 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386386    2570 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386389    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386391    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386394    2570 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386396    2570 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386399    2570 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386401    2570 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386404    2570 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386406    2570 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386409    2570 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:45:54.393551 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386412    2570 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386414    2570 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386417    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386420    2570 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386422    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386425    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386427    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386430    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386433    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386435    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386438    2570 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386440    2570 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386444    2570 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386448    2570 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386451    2570 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386454    2570 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386456    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386459    2570 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386461    2570 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:45:54.394012 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386464    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386466    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386469    2570 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386471    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386474    2570 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386476    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386479    2570 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386481    2570 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386484    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386486    2570 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386488    2570 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386491    2570 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386493    2570 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386496    2570 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386498    2570 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386501    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386504    2570 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386507    2570 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386510    2570 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:45:54.394455 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386512    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386515    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386518    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386520    2570 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386523    2570 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386525    2570 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386528    2570 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386530    2570 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386533    2570 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386535    2570 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386538    2570 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386540    2570 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386543    2570 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386546    2570 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386548    2570 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386551    2570 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386553    2570 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386555    2570 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386558    2570 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386560    2570 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:45:54.394911 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386563    2570 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386565    2570 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386568    2570 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386570    2570 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386573    2570 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386586    2570 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386589    2570 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386591    2570 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386594    2570 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386597    2570 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386599    2570 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386602    2570 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386604    2570 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386608    2570 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:54.386610    2570 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.386662    2570 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.388816    2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.389958    2570 server.go:1014] "Starting client certificate rotation"
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.390053    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:45:54.395381 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.390094    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:45:54.396138 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.392330    2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:45:54.402261 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.402244    2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:45:54.415455 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.415438    2570 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:45:54.421600 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.421570    2570 log.go:25] "Validated CRI v1 image API"
Oct 30 18:45:54.422435 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.422409    2570 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:45:54.425330 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.425302    2570 fs.go:135] Filesystem UUIDs: map[74da84f2-0b61-4b9f-b60a-82c89b126057:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cfbc3d41-e7f3-4785-8e52-0150bb65b235:/dev/nvme0n1p3]
Oct 30 18:45:54.425413 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.425329    2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}]
Oct 30 18:45:54.431230 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.431111    2570 manager.go:217] Machine: {Timestamp:2026-10-30 18:45:54.428963517 +0000 UTC m=+0.410102007 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101640 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26c4b2fb4db98a9fb9b885e5d11722 SystemUUID:ec26c4b2-fb4d-b98a-9fb9-b885e5d11722 BootID:1fc1392e-c7ec-4db6-94a9-393ed7a98721 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:06:ff:f9:9e:44:11 Speed:0 Mtu:9001} {Name:ens5 MacAddress:06:ff:f9:9e:44:11 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:16:b0:c3:a6:84 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:45:54.431230 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.431226    2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:45:54.431392 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.431364    2570 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:45:54.435049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.435022    2570 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:45:54.435170 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.435051    2570 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-33.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:45:54.436814 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.436803    2570 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:45:54.436855 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.436816    2570 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:45:54.436855 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.436843    2570 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:45:54.436910 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.436855    2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:45:54.437907 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.437896    2570 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:45:54.441944 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.441932    2570 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:45:54.442589 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.442571    2570 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:45:54.443543 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.443533    2570 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:45:54.443604 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.443554    2570 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:45:54.443604 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.443566    2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:45:54.443604 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.443575    2570 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:45:54.443604 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.443592    2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:45:54.443867 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.443852    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:45:54.446931 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.446910    2570 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:45:54.452722 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.452707    2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:45:54.453998 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.453985    2570 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:45:54.454058 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.454016    2570 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:45:54.454187 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.454175    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:45:54.454234 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.454197    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:45:54.454234 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.454210    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:45:54.454234 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.454217    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:45:54.454234 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.454232    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:45:54.454340 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.454245    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:45:54.455598 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.455588    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:45:54.455642 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.455599    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:45:54.455642 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.455615    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:45:54.455642 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.455622    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:45:54.455642 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.455630    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:45:54.455642 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.455638    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:45:54.456441 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.456431    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:45:54.456441 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.456441    2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:45:54.460090 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.460077    2570 server.go:1263] "Started kubelet"
Oct 30 18:45:54.460198 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.460165    2570 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:45:54.460283 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.460228    2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:45:54.460321 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.460308    2570 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:45:54.460797 ip-10-0-139-33 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:45:54.466533 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.466460    2570 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:45:54.469435 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.469032    2570 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:45:54.471668 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:54.471645    2570 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Oct 30 18:45:54.475190 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.475176    2570 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:45:54.475190 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.475184    2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:45:54.475851 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:54.475451    2570 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-33.us-west-2.compute.internal\" not found"
Oct 30 18:45:54.476120 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.476099    2570 factory.go:55] Registering systemd factory
Oct 30 18:45:54.476120 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.476118    2570 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:45:54.476305 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.476160    2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:45:54.476305 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.476273    2570 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:45:54.476305 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.476287    2570 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:45:54.476464 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.476372    2570 factory.go:153] Registering CRI-O factory
Oct 30 18:45:54.476464 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.476391    2570 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:45:54.476464 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.476419    2570 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:45:54.476464 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.476426    2570 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:45:54.476464 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.476439    2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:45:54.476464 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.476460    2570 factory.go:103] Registering Raw factory
Oct 30 18:45:54.476675 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.476475    2570 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:45:54.477615 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.477600    2570 manager.go:319] Starting recovery of all containers
Oct 30 18:45:54.489722 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.489708    2570 manager.go:324] Recovery completed
Oct 30 18:45:54.493433 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.493420    2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:54.494894 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.494877    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:54.494969 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.494904    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:54.494969 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.494916    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:54.495402 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.495388    2570 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:45:54.495402 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.495400    2570 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:45:54.495530 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.495409    2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:54.495530 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.495415    2570 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:45:54.496026 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.496012    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:54.496098 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.496043    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:54.496098 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.496059    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:54.497471 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.497458    2570 policy_none.go:50] "Start"
Oct 30 18:45:54.497528 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.497481    2570 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:45:54.497528 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.497491    2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:45:54.499394 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.499381    2570 policy_none.go:44] "Start"
Oct 30 18:45:54.541216 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.541079    2570 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:45:54.545031 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.545020    2570 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:45:54.545071 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:54.545049    2570 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:45:54.545071 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.545058    2570 server.go:87] "Starting device plugin registration server"
Oct 30 18:45:54.545293 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.545282    2570 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:45:54.545325 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.545296    2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:45:54.545412 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.545394    2570 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:45:54.545514 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.545496    2570 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:45:54.545596 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.545515    2570 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:45:54.546077 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:54.546059    2570 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:45:54.546162 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:54.546099    2570 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-33.us-west-2.compute.internal\" not found"
Oct 30 18:45:54.607764 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.607715    2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-33.us-west-2.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Oct 30 18:45:54.607764 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:54.607720    2570 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-139-33.us-west-2.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Oct 30 18:45:54.609702 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:54.607722    2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-33.us-west-2.compute.internal.18e3630618c4341d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-33.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-33.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:45:54.460054557 +0000 UTC m=+0.441193047,LastTimestamp:2026-10-30 18:45:54.460054557 +0000 UTC m=+0.441193047,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-33.us-west-2.compute.internal,}"
Oct 30 18:45:54.626414 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.626388    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:45:54.627500 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.627480    2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:45:54.627608 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.627507    2570 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:45:54.627608 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.627525    2570 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:45:54.627608 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:54.627561    2570 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Oct 30 18:45:54.646080 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.646064    2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:54.646784 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.646771    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:54.646851 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.646796    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:54.646851 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.646808    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:54.646851 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.646833    2570 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:54.682535 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.682516    2570 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:45:54.682605 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.682568    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-dtggm"
Oct 30 18:45:54.728305 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.728280    2570 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"]
Oct 30 18:45:54.728383 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.728368    2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:54.729166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.729151    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:54.729230 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.729177    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:54.729230 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.729187    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:54.731269 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.731257    2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:54.731428 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.731412    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:54.731497 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.731451    2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:54.731977 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.731960    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:54.732061 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.731988    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:54.732061 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.732004    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:54.732061 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.732054    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:54.732226 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.732079    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:54.732226 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.732094    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:54.734180 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.734165    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:54.734260 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.734192    2570 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:45:54.734842 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.734829    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:45:54.734904 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.734853    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:45:54.734904 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.734865    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:45:54.754869 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:54.754854    2570 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-33.us-west-2.compute.internal\" not found" node="ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:54.757416 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:54.757402    2570 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-33.us-west-2.compute.internal\" not found" node="ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:54.763626 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.763613    2570 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:45:54.778320 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.778302    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91959f1cc4ebe49096831f4c0e77120d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"91959f1cc4ebe49096831f4c0e77120d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:54.778396 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.778327    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a61da0484b7ed0271380ee2068515a4-config\") pod \"kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"9a61da0484b7ed0271380ee2068515a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:54.778396 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:54.778342    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91959f1cc4ebe49096831f4c0e77120d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"91959f1cc4ebe49096831f4c0e77120d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:56.614978 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.614927    2570 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:45:56.616657 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.616635    2570 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:45:56.617208 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.617193    2570 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:45:56.618222 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.618207    2570 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:56.618267 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:56.618229    2570 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-33.us-west-2.compute.internal\": node \"ip-10-0-139-33.us-west-2.compute.internal\" not found"
Oct 30 18:45:56.634408 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.634388    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91959f1cc4ebe49096831f4c0e77120d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"91959f1cc4ebe49096831f4c0e77120d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:56.634452 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.634420    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91959f1cc4ebe49096831f4c0e77120d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"91959f1cc4ebe49096831f4c0e77120d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:56.634452 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.634437    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a61da0484b7ed0271380ee2068515a4-config\") pod \"kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"9a61da0484b7ed0271380ee2068515a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:56.634517 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.634479    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a61da0484b7ed0271380ee2068515a4-config\") pod \"kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"9a61da0484b7ed0271380ee2068515a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:56.634517 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.634486    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91959f1cc4ebe49096831f4c0e77120d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"91959f1cc4ebe49096831f4c0e77120d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:56.634517 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.634501    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91959f1cc4ebe49096831f4c0e77120d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"91959f1cc4ebe49096831f4c0e77120d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:56.731260 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.731243    2570 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:56.816489 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.816465    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:56.818621 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.818605    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:56.848236 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.848220    2570 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:45:56.848343 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.848329    2570 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:45:56.914914 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:56.914872    2570 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:45:57.145020 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.144992    2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Oct 30 18:45:57.145485 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.145116    2570 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 18:45:57.145485 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.145119    2570 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 18:45:57.145485 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.145127    2570 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received"
Oct 30 18:45:57.145485 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:57.145121    2570 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://a08f964132a324d829023bb7d5fff707-ee203c489136d9c1.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-139-33.us-west-2.compute.internal.18e363061ad834e9\": read tcp 10.0.139.33:48150->18.224.117.112:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-139-33.us-west-2.compute.internal.18e363061ad834e9  default   5691 0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-33.us-west-2.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-139-33.us-west-2.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-139-33.us-west-2.compute.internal,},FirstTimestamp:2026-10-30 18:45:54 +0000 UTC,LastTimestamp:2026-10-30 18:45:54.646811746 +0000 UTC m=+0.627950236,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-33.us-west-2.compute.internal,}"
Oct 30 18:45:57.199880 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.199834    2570 apiserver.go:52] "Watching apiserver"
Oct 30 18:45:57.231544 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.231529    2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Oct 30 18:45:57.439669 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.439645    2570 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:45:57.439941 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.439923    2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal","openshift-multus/multus-vkvjm","openshift-multus/multus-additional-cni-plugins-ph92c","openshift-multus/network-metrics-daemon-pgbfm","openshift-network-diagnostics/network-check-target-42cjm","openshift-network-operator/iptables-alerter-v6t5q","openshift-ovn-kubernetes/ovnkube-node-45wjp","kube-system/global-pull-secret-syncer-wpm6z","kube-system/konnectivity-agent-nv7ql","openshift-cluster-node-tuning-operator/tuned-jddc7","kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx","openshift-dns/node-resolver-hx8pq","openshift-image-registry/node-ca-664td"]
Oct 30 18:45:57.442247 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.442233    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.443043 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.442996    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:40:56 +0000 UTC" deadline="2028-08-05 16:39:46.752133928 +0000 UTC"
Oct 30 18:45:57.443043 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.443029    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15477h53m49.30910868s"
Oct 30 18:45:57.444401 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.444387    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.446334 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.446321    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:45:57.446410 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:57.446384    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:45:57.448265 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.448251    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:45:57.448326 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:57.448311    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:45:57.450333 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.450303    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:45:57.452371 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.452357    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.454462 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.454446    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:45:57.454548 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:57.454504    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:45:57.456563 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.456545    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:45:57.458685 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.458667    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.460837 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.460812    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.462938 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.462922    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:45:57.465152 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.465134    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-664td"
Oct 30 18:45:57.488202 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.488185    2570 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Oct 30 18:45:57.488275 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.488231    2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w27ff"
Oct 30 18:45:57.491241 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.491219    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:45:57.491241 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.491219    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:45:57.491418 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.491369    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:45:57.491418 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.491393    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:45:57.491418 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.491409    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-56wtl\""
Oct 30 18:45:57.492752 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.492738    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:45:57.495795 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.495782    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bpjwm\""
Oct 30 18:45:57.496183 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.496165    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:45:57.502975 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.502959    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-77vpl\""
Oct 30 18:45:57.512447 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.512432    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-r47g8\""
Oct 30 18:45:57.513168 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.513154    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:45:57.513496 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.513485    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:45:57.514143 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.514127    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:45:57.514204 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.514162    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-29qlm\""
Oct 30 18:45:57.514204 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.514180    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:45:57.514299 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.514223    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:45:57.514362 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.514349    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:45:57.514470 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.514458    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:45:57.532774 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.532755    2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:45:57.534634 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.534616    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:45:57.534713 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.534660    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:45:57.534713 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.534667    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:45:57.534713 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.534660    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:45:57.537886 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.537870    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:45:57.538194 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538179    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/652a5a58-ad99-49bb-b371-743c2816482d-ovn-node-metrics-cert\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.538249 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538202    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-kubernetes\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.538249 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538221    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-multus-certs\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.538344 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538246    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77jh2\" (UniqueName: \"kubernetes.io/projected/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-kube-api-access-77jh2\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.538399 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538373    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nck5c\" (UniqueName: \"kubernetes.io/projected/9822bfc7-212f-4cf6-8d56-eb44c58c359b-kube-api-access-nck5c\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:45:57.538439 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538423    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysctl-d\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.538484 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538450    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-tuned\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.538484 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538475    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.538543 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538499    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrrbp\" (UniqueName: \"kubernetes.io/projected/4cb022e6-43bd-4d71-9935-f0e91130735b-kube-api-access-vrrbp\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:45:57.538543 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538524    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/738e26c2-0aac-4df0-961d-8c7404c8d6ab-hosts-file\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:45:57.538613 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538557    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-slash\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.538613 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538591    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cnibin\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.538683 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538622    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-var-lib-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.538683 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538651    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-node-log\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.538746 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538691    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-system-cni-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.538746 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538727    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:45:57.538813 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538745    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-lib-modules\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.538813 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538763    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-host\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.538813 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538792    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-registration-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.538922 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538828    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-env-overrides\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.538922 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538851    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2g7p\" (UniqueName: \"kubernetes.io/projected/652a5a58-ad99-49bb-b371-743c2816482d-kube-api-access-r2g7p\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.538922 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538883    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-sys\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.539017 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538922    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11550a43-fc10-43e9-835c-ae7c8b21be03-tmp\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.539017 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538952    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5229t\" (UniqueName: \"kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t\") pod \"network-check-target-42cjm\" (UID: \"ddbd05bb-e311-4a16-af9e-0ab5fa5103c0\") " pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:45:57.539017 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.538979    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-device-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.539017 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539005    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6fjw\" (UniqueName: \"kubernetes.io/projected/738e26c2-0aac-4df0-961d-8c7404c8d6ab-kube-api-access-w6fjw\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:45:57.539158 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539047    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9822bfc7-212f-4cf6-8d56-eb44c58c359b-serviceca\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:45:57.539158 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539086    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a6c94923-1962-425d-8e53-ad523e78619b-konnectivity-ca\") pod \"konnectivity-agent-nv7ql\" (UID: \"a6c94923-1962-425d-8e53-ad523e78619b\") " pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:45:57.539158 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539128    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-var-lib-kubelet\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.539158 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539154    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-kubelet\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.539279 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539171    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvkcs\" (UniqueName: \"kubernetes.io/projected/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-kube-api-access-pvkcs\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.539279 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539192    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-socket-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.539279 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539217    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghc57\" (UniqueName: \"kubernetes.io/projected/ea684836-96a5-4d68-a414-c3db5de7564e-kube-api-access-ghc57\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:45:57.539279 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539249    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-run-ovn-kubernetes\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.539279 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539267    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-cni-netd\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.539446 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539293    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-ovnkube-script-lib\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.539446 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539344    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-cni-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.539446 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539377    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-conf-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.539446 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539395    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg9z4\" (UniqueName: \"kubernetes.io/projected/cc73f4b8-ee7e-4915-a883-e26fa87a89db-kube-api-access-hg9z4\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.539446 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539411    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-systemd-units\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.539446 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539433    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.539630 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539459    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-systemd\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.539630 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539482    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-cni-bin\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.539630 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539514    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.539630 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539540    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-etc-selinux\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.539630 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539560    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-sys-fs\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.539630 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539576    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-run-netns\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.539630 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539603    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-cni-bin\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.539847 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539631    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-ovnkube-config\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.539847 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539651    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-run\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.539847 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539665    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdzp8\" (UniqueName: \"kubernetes.io/projected/11550a43-fc10-43e9-835c-ae7c8b21be03-kube-api-access-zdzp8\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.539847 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539707    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-cni-multus\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.539847 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539723    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea684836-96a5-4d68-a414-c3db5de7564e-host-slash\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:45:57.539847 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539747    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a6c94923-1962-425d-8e53-ad523e78619b-agent-certs\") pod \"konnectivity-agent-nv7ql\" (UID: \"a6c94923-1962-425d-8e53-ad523e78619b\") " pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:45:57.539847 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539768    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-cnibin\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.539847 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539810    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-etc-kubernetes\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.540189 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539850    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ea684836-96a5-4d68-a414-c3db5de7564e-iptables-alerter-script\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:45:57.540189 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539871    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-kubelet\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.540189 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539898    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b67fee5e-0972-4cfc-987d-b0ae79e6589d-dbus\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:45:57.540189 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539962    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:45:57.540189 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.539997    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysctl-conf\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.540189 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540023    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-socket-dir-parent\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.540189 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540088    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.540189 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540135    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc73f4b8-ee7e-4915-a883-e26fa87a89db-cni-binary-copy\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.540189 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540164    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-k8s-cni-cncf-io\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.540504 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540190    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-netns\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.540504 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540224    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/738e26c2-0aac-4df0-961d-8c7404c8d6ab-tmp-dir\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:45:57.540504 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540249    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b67fee5e-0972-4cfc-987d-b0ae79e6589d-kubelet-config\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:45:57.540504 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540263    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysconfig\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.540504 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540282    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-daemon-config\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.540504 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540319    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cni-binary-copy\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.540504 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540356    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9822bfc7-212f-4cf6-8d56-eb44c58c359b-host\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:45:57.540504 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540412    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-systemd\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.540504 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540440    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-etc-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.540504 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540460    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-log-socket\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.540504 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540505    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.540884 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540533    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.540884 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540603    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-modprobe-d\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.540884 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540668    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-hostroot\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.540884 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540699    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-system-cni-dir\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.540884 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.540721    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-ovn\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.543826 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.543812    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n4vwp\""
Oct 30 18:45:57.543882 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.543856    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5tsjg\""
Oct 30 18:45:57.544572 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.544561    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hz6tx\""
Oct 30 18:45:57.544630 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.544619    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lj6ts\""
Oct 30 18:45:57.544864 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.544852    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:45:57.546655 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.546644    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:45:57.546881 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.546870    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:45:57.546918 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.546885    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:45:57.546918 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.546912    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:45:57.546992 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.546982    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:45:57.547194 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.547183    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:45:57.547239 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.547225    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:45:57.548315 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.548301    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:45:57.641714 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641690    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-ovnkube-script-lib\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.641811 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641717    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-cni-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.641811 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641734    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-conf-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.641811 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641748    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg9z4\" (UniqueName: \"kubernetes.io/projected/cc73f4b8-ee7e-4915-a883-e26fa87a89db-kube-api-access-hg9z4\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.641811 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641765    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-systemd-units\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.641811 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641785    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.641811 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641806    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-systemd\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641812    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-conf-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641828    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-cni-bin\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641851    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641860    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-systemd-units\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641875    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-etc-selinux\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641869    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641900    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-sys-fs\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641910    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-cni-bin\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641925    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-run-netns\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641946    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-systemd\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641947    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-cni-bin\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641981    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-cni-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641997    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-ovnkube-config\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.641988    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-cni-bin\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642008    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-etc-selinux\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642023    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-run\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642049    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-sys-fs\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.642166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642077    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-run\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642094    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-run-netns\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642075    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdzp8\" (UniqueName: \"kubernetes.io/projected/11550a43-fc10-43e9-835c-ae7c8b21be03-kube-api-access-zdzp8\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642131    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-cni-multus\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642154    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642173    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea684836-96a5-4d68-a414-c3db5de7564e-host-slash\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642189    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a6c94923-1962-425d-8e53-ad523e78619b-agent-certs\") pod \"konnectivity-agent-nv7ql\" (UID: \"a6c94923-1962-425d-8e53-ad523e78619b\") " pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642200    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-cni-multus\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642210    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-cnibin\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642247    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea684836-96a5-4d68-a414-c3db5de7564e-host-slash\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642263    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-etc-kubernetes\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642287    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ea684836-96a5-4d68-a414-c3db5de7564e-iptables-alerter-script\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642313    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-kubelet\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642320    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-etc-kubernetes\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642336    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b67fee5e-0972-4cfc-987d-b0ae79e6589d-dbus\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642341    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-cnibin\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642359    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642377    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-ovnkube-script-lib\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.643010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642404    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-kubelet\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642442    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b67fee5e-0972-4cfc-987d-b0ae79e6589d-dbus\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642385    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysctl-conf\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642478    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-socket-dir-parent\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642501    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:57.642507    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642532    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysctl-conf\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642525    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc73f4b8-ee7e-4915-a883-e26fa87a89db-cni-binary-copy\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642572    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-socket-dir-parent\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:57.642586    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret podName:b67fee5e-0972-4cfc-987d-b0ae79e6589d nodeName:}" failed. No retries permitted until 2026-10-30 18:45:58.14256715 +0000 UTC m=+2.368251197 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret") pod "global-pull-secret-syncer-wpm6z" (UID: "b67fee5e-0972-4cfc-987d-b0ae79e6589d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642574    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-ovnkube-config\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642575    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-k8s-cni-cncf-io\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642628    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642606    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-k8s-cni-cncf-io\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642650    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-netns\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642500    2570 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642684    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-netns\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642674    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/738e26c2-0aac-4df0-961d-8c7404c8d6ab-tmp-dir\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:45:57.643845 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642721    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b67fee5e-0972-4cfc-987d-b0ae79e6589d-kubelet-config\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642745    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysconfig\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642769    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-daemon-config\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642791    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cni-binary-copy\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642816    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9822bfc7-212f-4cf6-8d56-eb44c58c359b-host\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642835    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysconfig\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642789    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ea684836-96a5-4d68-a414-c3db5de7564e-iptables-alerter-script\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642858    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-systemd\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642883    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-etc-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.642940    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/738e26c2-0aac-4df0-961d-8c7404c8d6ab-tmp-dir\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643005    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-systemd\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643060    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc73f4b8-ee7e-4915-a883-e26fa87a89db-cni-binary-copy\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643065    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-etc-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643104    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b67fee5e-0972-4cfc-987d-b0ae79e6589d-kubelet-config\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643198    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-log-socket\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643317    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-daemon-config\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643357    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cni-binary-copy\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643451    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9822bfc7-212f-4cf6-8d56-eb44c58c359b-host\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:45:57.644665 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643480    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-log-socket\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643513    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643549    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643574    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-modprobe-d\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643619    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643656    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-hostroot\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643676    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-modprobe-d\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643683    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-system-cni-dir\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643714    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-ovn\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643716    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-system-cni-dir\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643747    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-ovn\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643755    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-hostroot\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643741    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/652a5a58-ad99-49bb-b371-743c2816482d-ovn-node-metrics-cert\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643801    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-kubernetes\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643826    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-multus-certs\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643851    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77jh2\" (UniqueName: \"kubernetes.io/projected/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-kube-api-access-77jh2\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643870    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-kubernetes\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.645547 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643876    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nck5c\" (UniqueName: \"kubernetes.io/projected/9822bfc7-212f-4cf6-8d56-eb44c58c359b-kube-api-access-nck5c\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643907    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-multus-certs\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643932    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysctl-d\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643956    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-tuned\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.643989    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644017    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrrbp\" (UniqueName: \"kubernetes.io/projected/4cb022e6-43bd-4d71-9935-f0e91130735b-kube-api-access-vrrbp\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644055    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/738e26c2-0aac-4df0-961d-8c7404c8d6ab-hosts-file\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644078    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-slash\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644126    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-slash\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644016    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644156    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysctl-d\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644192    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/738e26c2-0aac-4df0-961d-8c7404c8d6ab-hosts-file\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644221    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cnibin\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644246    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-var-lib-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644269    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-node-log\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644316    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-system-cni-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644324    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cnibin\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.646361 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644340    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644363    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-node-log\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644367    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-lib-modules\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644327    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-var-lib-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644399    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-host\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644404    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-system-cni-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644424    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-registration-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644451    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-env-overrides\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644475    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-lib-modules\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644477    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2g7p\" (UniqueName: \"kubernetes.io/projected/652a5a58-ad99-49bb-b371-743c2816482d-kube-api-access-r2g7p\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644506    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-sys\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:57.644501    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644535    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:57.644580    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs podName:4cb022e6-43bd-4d71-9935-f0e91130735b nodeName:}" failed. No retries permitted until 2026-10-30 18:45:58.144564745 +0000 UTC m=+2.370248797 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs") pod "network-metrics-daemon-pgbfm" (UID: "4cb022e6-43bd-4d71-9935-f0e91130735b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644534    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11550a43-fc10-43e9-835c-ae7c8b21be03-tmp\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644609    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-sys\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644644    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5229t\" (UniqueName: \"kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t\") pod \"network-check-target-42cjm\" (UID: \"ddbd05bb-e311-4a16-af9e-0ab5fa5103c0\") " pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:45:57.647049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644657    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-registration-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644671    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-device-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644698    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-host\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644719    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6fjw\" (UniqueName: \"kubernetes.io/projected/738e26c2-0aac-4df0-961d-8c7404c8d6ab-kube-api-access-w6fjw\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644744    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9822bfc7-212f-4cf6-8d56-eb44c58c359b-serviceca\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644786    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a6c94923-1962-425d-8e53-ad523e78619b-konnectivity-ca\") pod \"konnectivity-agent-nv7ql\" (UID: \"a6c94923-1962-425d-8e53-ad523e78619b\") " pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644812    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-var-lib-kubelet\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644864    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-kubelet\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644918    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvkcs\" (UniqueName: \"kubernetes.io/projected/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-kube-api-access-pvkcs\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644946    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-socket-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644947    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-kubelet\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644972    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghc57\" (UniqueName: \"kubernetes.io/projected/ea684836-96a5-4d68-a414-c3db5de7564e-kube-api-access-ghc57\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644977    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-env-overrides\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.644998    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-run-ovn-kubernetes\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.645020    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-cni-netd\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.645021    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-device-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.645186    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-socket-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.647507 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.645321    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-var-lib-kubelet\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.647942 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.645345    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-run-ovn-kubernetes\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.647942 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.645366    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-cni-netd\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.647942 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.646017    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9822bfc7-212f-4cf6-8d56-eb44c58c359b-serviceca\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:45:57.647942 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.646175    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a6c94923-1962-425d-8e53-ad523e78619b-konnectivity-ca\") pod \"konnectivity-agent-nv7ql\" (UID: \"a6c94923-1962-425d-8e53-ad523e78619b\") " pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:45:57.647942 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.646829    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/652a5a58-ad99-49bb-b371-743c2816482d-ovn-node-metrics-cert\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.647942 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.646859    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a6c94923-1962-425d-8e53-ad523e78619b-agent-certs\") pod \"konnectivity-agent-nv7ql\" (UID: \"a6c94923-1962-425d-8e53-ad523e78619b\") " pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:45:57.647942 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.646929    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-tuned\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.647942 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.646977    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11550a43-fc10-43e9-835c-ae7c8b21be03-tmp\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.698971 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.698946    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg9z4\" (UniqueName: \"kubernetes.io/projected/cc73f4b8-ee7e-4915-a883-e26fa87a89db-kube-api-access-hg9z4\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.712284 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.712191    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6fjw\" (UniqueName: \"kubernetes.io/projected/738e26c2-0aac-4df0-961d-8c7404c8d6ab-kube-api-access-w6fjw\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:45:57.712394 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.712378    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nck5c\" (UniqueName: \"kubernetes.io/projected/9822bfc7-212f-4cf6-8d56-eb44c58c359b-kube-api-access-nck5c\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:45:57.712516 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.712498    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghc57\" (UniqueName: \"kubernetes.io/projected/ea684836-96a5-4d68-a414-c3db5de7564e-kube-api-access-ghc57\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:45:57.714109 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.714090    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvkcs\" (UniqueName: \"kubernetes.io/projected/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-kube-api-access-pvkcs\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.716186 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:57.716161    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:45:57.716334 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:57.716190    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:45:57.716334 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:57.716214    2570 projected.go:196] Error preparing data for projected volume kube-api-access-5229t for pod openshift-network-diagnostics/network-check-target-42cjm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:57.716334 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:57.716278    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t podName:ddbd05bb-e311-4a16-af9e-0ab5fa5103c0 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:58.21626005 +0000 UTC m=+2.441944109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5229t" (UniqueName: "kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t") pod "network-check-target-42cjm" (UID: "ddbd05bb-e311-4a16-af9e-0ab5fa5103c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:57.718597 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.718571    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77jh2\" (UniqueName: \"kubernetes.io/projected/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-kube-api-access-77jh2\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.718702 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.718611    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrrbp\" (UniqueName: \"kubernetes.io/projected/4cb022e6-43bd-4d71-9935-f0e91130735b-kube-api-access-vrrbp\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:45:57.720201 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.720183    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdzp8\" (UniqueName: \"kubernetes.io/projected/11550a43-fc10-43e9-835c-ae7c8b21be03-kube-api-access-zdzp8\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.722294 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.722277    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2g7p\" (UniqueName: \"kubernetes.io/projected/652a5a58-ad99-49bb-b371-743c2816482d-kube-api-access-r2g7p\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.764952 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.764931    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vkvjm"
Oct 30 18:45:57.807747 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.807710    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:45:57.813549 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.813531    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:45:57.841217 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.841199    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:45:57.843207 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.843187    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:45:57.857364 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.857341    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:45:57.866413 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.866393    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:45:57.884109 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.884085    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:45:57.888535 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.888520    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-664td"
Oct 30 18:45:57.967599 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:57.967535    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91959f1cc4ebe49096831f4c0e77120d.slice/crio-daa36dfdd6a1af71a65334aa018896fed1f9e008030b0ff3f29a8bcdf60f895b WatchSource:0}: Error finding container daa36dfdd6a1af71a65334aa018896fed1f9e008030b0ff3f29a8bcdf60f895b: Status 404 returned error can't find the container with id daa36dfdd6a1af71a65334aa018896fed1f9e008030b0ff3f29a8bcdf60f895b
Oct 30 18:45:57.968390 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:57.968359    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a61da0484b7ed0271380ee2068515a4.slice/crio-f55989fc59f099067d253917e0f1844949c4e949832e54d51fe440249fa95219 WatchSource:0}: Error finding container f55989fc59f099067d253917e0f1844949c4e949832e54d51fe440249fa95219: Status 404 returned error can't find the container with id f55989fc59f099067d253917e0f1844949c4e949832e54d51fe440249fa95219
Oct 30 18:45:57.972891 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:57.972874    2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:45:58.149644 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:58.149622    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:45:58.150169 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:58.149670    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:45:58.150169 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:58.149753    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:58.150169 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:58.149798    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret podName:b67fee5e-0972-4cfc-987d-b0ae79e6589d nodeName:}" failed. No retries permitted until 2026-10-30 18:45:59.149785928 +0000 UTC m=+3.375469966 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret") pod "global-pull-secret-syncer-wpm6z" (UID: "b67fee5e-0972-4cfc-987d-b0ae79e6589d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:58.150169 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:58.149754    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:58.150169 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:58.149834    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs podName:4cb022e6-43bd-4d71-9935-f0e91130735b nodeName:}" failed. No retries permitted until 2026-10-30 18:45:59.149823857 +0000 UTC m=+3.375507897 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs") pod "network-metrics-daemon-pgbfm" (UID: "4cb022e6-43bd-4d71-9935-f0e91130735b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:58.250771 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:58.250749    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5229t\" (UniqueName: \"kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t\") pod \"network-check-target-42cjm\" (UID: \"ddbd05bb-e311-4a16-af9e-0ab5fa5103c0\") " pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:45:58.250887 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:58.250846    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:45:58.250887 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:58.250857    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:45:58.250887 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:58.250867    2570 projected.go:196] Error preparing data for projected volume kube-api-access-5229t for pod openshift-network-diagnostics/network-check-target-42cjm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:58.250991 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:58.250906    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t podName:ddbd05bb-e311-4a16-af9e-0ab5fa5103c0 nodeName:}" failed. No retries permitted until 2026-10-30 18:45:59.250895579 +0000 UTC m=+3.476579617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5229t" (UniqueName: "kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t") pod "network-check-target-42cjm" (UID: "ddbd05bb-e311-4a16-af9e-0ab5fa5103c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:58.386806 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:58.386764    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal" event={"ID":"91959f1cc4ebe49096831f4c0e77120d","Type":"ContainerStarted","Data":"daa36dfdd6a1af71a65334aa018896fed1f9e008030b0ff3f29a8bcdf60f895b"}
Oct 30 18:45:58.387747 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:58.387726    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal" event={"ID":"9a61da0484b7ed0271380ee2068515a4","Type":"ContainerStarted","Data":"f55989fc59f099067d253917e0f1844949c4e949832e54d51fe440249fa95219"}
Oct 30 18:45:58.491685 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:58.491646    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:40:57 +0000 UTC" deadline="2028-04-10 13:04:38.539187616 +0000 UTC"
Oct 30 18:45:58.491685 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:58.491681    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12666h18m40.047508565s"
Oct 30 18:45:58.543197 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:58.543145    2570 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:45:58.626168 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:58.626141    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6c94923_1962_425d_8e53_ad523e78619b.slice/crio-2c86a6df7933fd5687cc84bb4c30258050ccf42bd55d0f50f7fcdc89fc7f428d WatchSource:0}: Error finding container 2c86a6df7933fd5687cc84bb4c30258050ccf42bd55d0f50f7fcdc89fc7f428d: Status 404 returned error can't find the container with id 2c86a6df7933fd5687cc84bb4c30258050ccf42bd55d0f50f7fcdc89fc7f428d
Oct 30 18:45:58.644776 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:58.644743    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea684836_96a5_4d68_a414_c3db5de7564e.slice/crio-de7265384d17948ad084102cbb4ca2c97b44c67b5a062aba3e6c16d511b8d2a1 WatchSource:0}: Error finding container de7265384d17948ad084102cbb4ca2c97b44c67b5a062aba3e6c16d511b8d2a1: Status 404 returned error can't find the container with id de7265384d17948ad084102cbb4ca2c97b44c67b5a062aba3e6c16d511b8d2a1
Oct 30 18:45:58.651108 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:58.651090    2570 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:45:58.686524 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:58.686506    2570 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:45:58.690488 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:58.690465    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cbb0cdd_8ae2_4149_84d8_5df55b50e54a.slice/crio-d3e3fcdbe90125127b344c0c4a23486da5bef2f20e0c7df390a87b46b0c261d9 WatchSource:0}: Error finding container d3e3fcdbe90125127b344c0c4a23486da5bef2f20e0c7df390a87b46b0c261d9: Status 404 returned error can't find the container with id d3e3fcdbe90125127b344c0c4a23486da5bef2f20e0c7df390a87b46b0c261d9
Oct 30 18:45:58.716909 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:58.716887    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc73f4b8_ee7e_4915_a883_e26fa87a89db.slice/crio-4411f88293ac8985be9ff0bf0c171ca23852e5509dc762a19cce13c841ae8c4d WatchSource:0}: Error finding container 4411f88293ac8985be9ff0bf0c171ca23852e5509dc762a19cce13c841ae8c4d: Status 404 returned error can't find the container with id 4411f88293ac8985be9ff0bf0c171ca23852e5509dc762a19cce13c841ae8c4d
Oct 30 18:45:58.734570 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:58.734545    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcda0bc7c_777f_4965_b097_6b5f3b6bfc52.slice/crio-69f9ba226e0a06bb9ced9993a88533d9b4a09b10eb66144081a213f1390cfed8 WatchSource:0}: Error finding container 69f9ba226e0a06bb9ced9993a88533d9b4a09b10eb66144081a213f1390cfed8: Status 404 returned error can't find the container with id 69f9ba226e0a06bb9ced9993a88533d9b4a09b10eb66144081a213f1390cfed8
Oct 30 18:45:58.756327 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:58.756307    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9822bfc7_212f_4cf6_8d56_eb44c58c359b.slice/crio-6b9097d4a86d0d7d8cd8544da83af7d44980b474ef763e394400775fe3d1538e WatchSource:0}: Error finding container 6b9097d4a86d0d7d8cd8544da83af7d44980b474ef763e394400775fe3d1538e: Status 404 returned error can't find the container with id 6b9097d4a86d0d7d8cd8544da83af7d44980b474ef763e394400775fe3d1538e
Oct 30 18:45:58.790458 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:45:58.790439    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11550a43_fc10_43e9_835c_ae7c8b21be03.slice/crio-bfc617d19214966d7fd6017d82481aa1ab250ba8af96907378c52aec044f5f11 WatchSource:0}: Error finding container bfc617d19214966d7fd6017d82481aa1ab250ba8af96907378c52aec044f5f11: Status 404 returned error can't find the container with id bfc617d19214966d7fd6017d82481aa1ab250ba8af96907378c52aec044f5f11
Oct 30 18:45:59.156588 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.156547    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:45:59.157232 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.156638    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:45:59.157232 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:59.156771    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:59.157232 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:59.156859    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret podName:b67fee5e-0972-4cfc-987d-b0ae79e6589d nodeName:}" failed. No retries permitted until 2026-10-30 18:46:01.156821639 +0000 UTC m=+5.382505679 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret") pod "global-pull-secret-syncer-wpm6z" (UID: "b67fee5e-0972-4cfc-987d-b0ae79e6589d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:45:59.157232 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:59.156926    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:59.157232 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:59.156990    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs podName:4cb022e6-43bd-4d71-9935-f0e91130735b nodeName:}" failed. No retries permitted until 2026-10-30 18:46:01.156977492 +0000 UTC m=+5.382661533 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs") pod "network-metrics-daemon-pgbfm" (UID: "4cb022e6-43bd-4d71-9935-f0e91130735b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:45:59.257854 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.257819    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5229t\" (UniqueName: \"kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t\") pod \"network-check-target-42cjm\" (UID: \"ddbd05bb-e311-4a16-af9e-0ab5fa5103c0\") " pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:45:59.258044 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:59.258009    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:45:59.258113 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:59.258049    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:45:59.258113 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:59.258063    2570 projected.go:196] Error preparing data for projected volume kube-api-access-5229t for pod openshift-network-diagnostics/network-check-target-42cjm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:59.258210 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:59.258121    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t podName:ddbd05bb-e311-4a16-af9e-0ab5fa5103c0 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:01.25810411 +0000 UTC m=+5.483788167 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5229t" (UniqueName: "kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t") pod "network-check-target-42cjm" (UID: "ddbd05bb-e311-4a16-af9e-0ab5fa5103c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:45:59.383594 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.383563    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:45:59.383767 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:59.383670    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:45:59.383767 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.383755    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:45:59.383881 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:59.383838    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:45:59.383934 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.383899    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:45:59.383984 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:45:59.383966    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:45:59.391020 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.390986    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vkvjm" event={"ID":"cc73f4b8-ee7e-4915-a883-e26fa87a89db","Type":"ContainerStarted","Data":"4411f88293ac8985be9ff0bf0c171ca23852e5509dc762a19cce13c841ae8c4d"}
Oct 30 18:45:59.392369 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.392133    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v6t5q" event={"ID":"ea684836-96a5-4d68-a414-c3db5de7564e","Type":"ContainerStarted","Data":"de7265384d17948ad084102cbb4ca2c97b44c67b5a062aba3e6c16d511b8d2a1"}
Oct 30 18:45:59.393444 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.393419    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nv7ql" event={"ID":"a6c94923-1962-425d-8e53-ad523e78619b","Type":"ContainerStarted","Data":"2c86a6df7933fd5687cc84bb4c30258050ccf42bd55d0f50f7fcdc89fc7f428d"}
Oct 30 18:45:59.394831 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.394806    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hx8pq" event={"ID":"738e26c2-0aac-4df0-961d-8c7404c8d6ab","Type":"ContainerStarted","Data":"ab2f241cbd27f93527f68f203a29005c910414f14a4eeedc9e501ddbba116f98"}
Oct 30 18:45:59.396197 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.396162    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp" event={"ID":"652a5a58-ad99-49bb-b371-743c2816482d","Type":"ContainerStarted","Data":"17bf4ea6d518f6a340d2f3cc1e666c8fd718ff015e8c61fb7cb9ac843d44273c"}
Oct 30 18:45:59.397449 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.397431    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ph92c" event={"ID":"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a","Type":"ContainerStarted","Data":"d3e3fcdbe90125127b344c0c4a23486da5bef2f20e0c7df390a87b46b0c261d9"}
Oct 30 18:45:59.398820 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.398791    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jddc7" event={"ID":"11550a43-fc10-43e9-835c-ae7c8b21be03","Type":"ContainerStarted","Data":"bfc617d19214966d7fd6017d82481aa1ab250ba8af96907378c52aec044f5f11"}
Oct 30 18:45:59.400057 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.400019    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-664td" event={"ID":"9822bfc7-212f-4cf6-8d56-eb44c58c359b","Type":"ContainerStarted","Data":"6b9097d4a86d0d7d8cd8544da83af7d44980b474ef763e394400775fe3d1538e"}
Oct 30 18:45:59.401257 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.401239    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx" event={"ID":"cda0bc7c-777f-4965-b097-6b5f3b6bfc52","Type":"ContainerStarted","Data":"69f9ba226e0a06bb9ced9993a88533d9b4a09b10eb66144081a213f1390cfed8"}
Oct 30 18:45:59.492153 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.492060    2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:40:57 +0000 UTC" deadline="2028-07-13 04:26:54.415177796 +0000 UTC"
Oct 30 18:45:59.492153 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:45:59.492114    2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14913h40m54.923067302s"
Oct 30 18:46:01.174916 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:01.174649    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:01.175411 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:01.174948    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:01.175411 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:01.174805    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:46:01.175411 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:01.175045    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret podName:b67fee5e-0972-4cfc-987d-b0ae79e6589d nodeName:}" failed. No retries permitted until 2026-10-30 18:46:05.175009542 +0000 UTC m=+9.400693594 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret") pod "global-pull-secret-syncer-wpm6z" (UID: "b67fee5e-0972-4cfc-987d-b0ae79e6589d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:46:01.175411 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:01.175093    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:46:01.175411 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:01.175152    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs podName:4cb022e6-43bd-4d71-9935-f0e91130735b nodeName:}" failed. No retries permitted until 2026-10-30 18:46:05.175134434 +0000 UTC m=+9.400818488 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs") pod "network-metrics-daemon-pgbfm" (UID: "4cb022e6-43bd-4d71-9935-f0e91130735b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:46:01.275884 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:01.275850    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5229t\" (UniqueName: \"kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t\") pod \"network-check-target-42cjm\" (UID: \"ddbd05bb-e311-4a16-af9e-0ab5fa5103c0\") " pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:01.276082 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:01.276002    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:46:01.276082 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:01.276024    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:46:01.276191 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:01.276100    2570 projected.go:196] Error preparing data for projected volume kube-api-access-5229t for pod openshift-network-diagnostics/network-check-target-42cjm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:46:01.276247 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:01.276189    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t podName:ddbd05bb-e311-4a16-af9e-0ab5fa5103c0 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:05.276169171 +0000 UTC m=+9.501853231 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5229t" (UniqueName: "kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t") pod "network-check-target-42cjm" (UID: "ddbd05bb-e311-4a16-af9e-0ab5fa5103c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:46:01.383761 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:01.383731    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:01.383944 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:01.383857    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:01.383944 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:01.383918    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:01.384107 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:01.383987    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:01.384107 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:01.384046    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:01.384216 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:01.384122    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:01.416766 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:01.416182    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal" event={"ID":"9a61da0484b7ed0271380ee2068515a4","Type":"ContainerStarted","Data":"b717f1f59d3109533ef9c11e85f3bb6f50643fe8d4d33c89dd17a814fe4dfb60"}
Oct 30 18:46:03.384251 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:03.384219    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:03.384682 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:03.384346    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:03.384682 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:03.384452    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:03.384682 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:03.384608    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:03.384682 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:03.384679    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:03.384897 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:03.384763    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:03.430166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:03.430105    2570 generic.go:356] "Generic (PLEG): container finished" podID="91959f1cc4ebe49096831f4c0e77120d" containerID="4cc911b2e82c35b1d86d640e48724dd1eabdab4039c18a15873a82f13a5e0954" exitCode=0
Oct 30 18:46:03.430290 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:03.430237    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal" event={"ID":"91959f1cc4ebe49096831f4c0e77120d","Type":"ContainerDied","Data":"4cc911b2e82c35b1d86d640e48724dd1eabdab4039c18a15873a82f13a5e0954"}
Oct 30 18:46:03.433191 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:03.433163    2570 generic.go:356] "Generic (PLEG): container finished" podID="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" containerID="7466e3999e642dbc919bd1c8fa65e533920be782f3bfb63b017308859b8f1336" exitCode=0
Oct 30 18:46:03.433293 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:03.433255    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ph92c" event={"ID":"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a","Type":"ContainerDied","Data":"7466e3999e642dbc919bd1c8fa65e533920be782f3bfb63b017308859b8f1336"}
Oct 30 18:46:03.437987 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:03.437965    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-nv7ql" event={"ID":"a6c94923-1962-425d-8e53-ad523e78619b","Type":"ContainerStarted","Data":"aff5a75c11b2de2a439e9c772cef2dfcbd483d81c2d491da96ee7acef41bbf95"}
Oct 30 18:46:03.540913 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:03.540856    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal" podStartSLOduration=7.540837165 podStartE2EDuration="7.540837165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:01.526159804 +0000 UTC m=+5.751843866" watchObservedRunningTime="2026-10-30 18:46:03.540837165 +0000 UTC m=+7.766521227"
Oct 30 18:46:04.070891 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:04.070831    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:04.445217 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:04.445146    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal" event={"ID":"91959f1cc4ebe49096831f4c0e77120d","Type":"ContainerStarted","Data":"d54e9cef42e70f225c3d005f8da05c9d0ef430eaa57ebb31abd21e9335987b04"}
Oct 30 18:46:04.562776 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:04.562716    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-nv7ql" podStartSLOduration=4.014300279 podStartE2EDuration="8.562694829s" totalImagesPullingTime="4.54839455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:03.78645603 +0000 UTC m=+8.012140095" watchObservedRunningTime="2026-10-30 18:46:04.562694829 +0000 UTC m=+8.788378881"
Oct 30 18:46:04.563257 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:04.563223    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal" podStartSLOduration=8.563209193 podStartE2EDuration="8.563209193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:04.562821326 +0000 UTC m=+8.788505387" watchObservedRunningTime="2026-10-30 18:46:04.563209193 +0000 UTC m=+8.788893255"
Oct 30 18:46:05.212740 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:05.212659    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:05.212740 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:05.212714    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:05.212959 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:05.212862    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:46:05.212959 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:05.212919    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs podName:4cb022e6-43bd-4d71-9935-f0e91130735b nodeName:}" failed. No retries permitted until 2026-10-30 18:46:13.21290212 +0000 UTC m=+17.438586177 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs") pod "network-metrics-daemon-pgbfm" (UID: "4cb022e6-43bd-4d71-9935-f0e91130735b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:46:05.213372 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:05.213308    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:46:05.213372 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:05.213359    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret podName:b67fee5e-0972-4cfc-987d-b0ae79e6589d nodeName:}" failed. No retries permitted until 2026-10-30 18:46:13.213345124 +0000 UTC m=+17.439029177 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret") pod "global-pull-secret-syncer-wpm6z" (UID: "b67fee5e-0972-4cfc-987d-b0ae79e6589d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:46:05.238944 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:05.238816    2570 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:46:05.239357 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:05.239337    2570 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:46:05.314186 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:05.313941    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5229t\" (UniqueName: \"kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t\") pod \"network-check-target-42cjm\" (UID: \"ddbd05bb-e311-4a16-af9e-0ab5fa5103c0\") " pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:05.314186 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:05.314097    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:46:05.314186 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:05.314110    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:46:05.314186 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:05.314119    2570 projected.go:196] Error preparing data for projected volume kube-api-access-5229t for pod openshift-network-diagnostics/network-check-target-42cjm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:46:05.314186 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:05.314156    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t podName:ddbd05bb-e311-4a16-af9e-0ab5fa5103c0 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:13.314144035 +0000 UTC m=+17.539828073 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5229t" (UniqueName: "kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t") pod "network-check-target-42cjm" (UID: "ddbd05bb-e311-4a16-af9e-0ab5fa5103c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:46:05.323537 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:05.323450    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:05.384059 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:05.383370    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:05.384059 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:05.383514    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:05.384059 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:05.383862    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:05.384059 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:05.383928    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:05.384059 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:05.383971    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:05.384059 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:05.384018    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:05.450352 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:05.450316    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-664td" event={"ID":"9822bfc7-212f-4cf6-8d56-eb44c58c359b","Type":"ContainerStarted","Data":"576ec03b67014035b1bf2e8579d7dd3c1cb61ab00a5d8b457688f8e9b7f2d033"}
Oct 30 18:46:05.454117 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:05.454081    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx" event={"ID":"cda0bc7c-777f-4965-b097-6b5f3b6bfc52","Type":"ContainerStarted","Data":"97a665095b443db47fad627dc5c5e8e158870384d12b39ad6edb736146b738f5"}
Oct 30 18:46:05.568879 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:05.568846    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:06.063841 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:06.063805    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:06.240666 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:06.240633    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:06.278607 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:06.278521    2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-664td_9822bfc7-212f-4cf6-8d56-eb44c58c359b/node-ca/0.log"
Oct 30 18:46:06.426800 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:06.426679    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:06.455654 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:06.455626    2570 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:46:06.853997 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:06.853963    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:07.084780 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.084751    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:07.172747 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.172700    2570 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:46:07.266962 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.266922    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:07.308058 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.307956    2570 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:46:07.172715536Z","UUID":"30d90f2f-e7b0-4117-8cab-86e093a13e7e","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:46:07.309668 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.309643    2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:46:07.309789 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.309679    2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:46:07.383252 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.383190    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:07.383390 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:07.383301    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:07.383390 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.383371    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:07.383503 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:07.383438    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:07.383563 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.383501    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:07.383614 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:07.383585    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:07.460116 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.460083    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx" event={"ID":"cda0bc7c-777f-4965-b097-6b5f3b6bfc52","Type":"ContainerStarted","Data":"567afbf219cdda3db7465687186f7f7e8ac16d9ceec82b613e00b89ad92c464d"}
Oct 30 18:46:07.462049 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.461989    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vkvjm" event={"ID":"cc73f4b8-ee7e-4915-a883-e26fa87a89db","Type":"ContainerStarted","Data":"d9162994fd60c49736f3da001c1a2928e1cede6952fc3b868a7359c3a423443f"}
Oct 30 18:46:07.464015 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.463990    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hx8pq" event={"ID":"738e26c2-0aac-4df0-961d-8c7404c8d6ab","Type":"ContainerStarted","Data":"5ca756c87c47771f385047f98147b9530320317a864588d0739d400a2dfac9b7"}
Oct 30 18:46:07.480156 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.480122    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:07.573573 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.573346    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-664td" podStartSLOduration=5.8586664840000005 podStartE2EDuration="11.573328247s" totalImagesPullingTime="5.714661763s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:05.559602695 +0000 UTC m=+9.785286756" watchObservedRunningTime="2026-10-30 18:46:07.573328247 +0000 UTC m=+11.799012308"
Oct 30 18:46:07.573743 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.573665    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-vkvjm" podStartSLOduration=3.856532117 podStartE2EDuration="11.573658961s" totalImagesPullingTime="7.717126844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:07.573167303 +0000 UTC m=+11.798851367" watchObservedRunningTime="2026-10-30 18:46:07.573658961 +0000 UTC m=+11.799343021"
Oct 30 18:46:07.652304 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.652229    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:07.681882 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:07.681838    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-hx8pq" podStartSLOduration=3.363971419 podStartE2EDuration="11.68182049s" totalImagesPullingTime="8.317849071s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:07.681184628 +0000 UTC m=+11.906868689" watchObservedRunningTime="2026-10-30 18:46:07.68182049 +0000 UTC m=+11.907504578"
Oct 30 18:46:08.472065 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:08.472017    2570 generic.go:356] "Generic (PLEG): container finished" podID="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" containerID="fa01b73e715d03a0f9fce5e59a379833c404fdedf4cded85d09cb898646eec63" exitCode=0
Oct 30 18:46:08.472533 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:08.472144    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ph92c" event={"ID":"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a","Type":"ContainerDied","Data":"fa01b73e715d03a0f9fce5e59a379833c404fdedf4cded85d09cb898646eec63"}
Oct 30 18:46:08.763022 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:08.762915    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:08.940532 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:08.940495    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:09.123488 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:09.123461    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:09.311890 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:09.311861    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:09.384078 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:09.383997    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:09.384201 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:09.384091    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:09.384201 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:09.384178    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:09.384201 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:09.384184    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:09.384325 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:09.384271    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:09.384364 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:09.384332    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:09.477580 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:09.477549    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx" event={"ID":"cda0bc7c-777f-4965-b097-6b5f3b6bfc52","Type":"ContainerStarted","Data":"df9e19dab71cb65a2dc192ab680576d7d6faf7297faf4f0f0cd618e3057d86d7"}
Oct 30 18:46:09.479656 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:09.479620    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-v6t5q" event={"ID":"ea684836-96a5-4d68-a414-c3db5de7564e","Type":"ContainerStarted","Data":"618550bd8ddc931e82d97f1c56a3338e68105dfc1fd7a904018c0d598bc8e715"}
Oct 30 18:46:09.481953 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:09.481924    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-jddc7" event={"ID":"11550a43-fc10-43e9-835c-ae7c8b21be03","Type":"ContainerStarted","Data":"f97c16caba66f280673063331684df89deac7d7e5a1ceef3c2b4de5db906c01a"}
Oct 30 18:46:09.492016 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:09.491996    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:09.591131 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:09.591086    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx" podStartSLOduration=3.957265123 podStartE2EDuration="13.591071494s" totalImagesPullingTime="9.633806371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:09.590909384 +0000 UTC m=+13.816593444" watchObservedRunningTime="2026-10-30 18:46:09.591071494 +0000 UTC m=+13.816755557"
Oct 30 18:46:09.664991 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:09.664964    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:09.705968 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:09.705928    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-v6t5q" podStartSLOduration=5.28688335 podStartE2EDuration="13.705914201s" totalImagesPullingTime="8.419030851s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:09.705819547 +0000 UTC m=+13.931503618" watchObservedRunningTime="2026-10-30 18:46:09.705914201 +0000 UTC m=+13.931598261"
Oct 30 18:46:09.817219 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:09.817157    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-jddc7" podStartSLOduration=3.740522864 podStartE2EDuration="13.817138229s" totalImagesPullingTime="10.076615365s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:09.816259593 +0000 UTC m=+14.041943650" watchObservedRunningTime="2026-10-30 18:46:09.817138229 +0000 UTC m=+14.042822289"
Oct 30 18:46:10.304139 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:10.304101    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:10.483610 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:10.483577    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:10.486159 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:10.486135    2570 generic.go:356] "Generic (PLEG): container finished" podID="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" containerID="92cc3dbffa14f0a3704766ad16b246d88e3772fa111bd7a657f64abc02949c6c" exitCode=0
Oct 30 18:46:10.486273 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:10.486252    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ph92c" event={"ID":"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a","Type":"ContainerDied","Data":"92cc3dbffa14f0a3704766ad16b246d88e3772fa111bd7a657f64abc02949c6c"}
Oct 30 18:46:10.733991 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:10.733909    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:10.941115 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:10.941085    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:11.132867 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:11.132836    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:11.337431 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:11.337402    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:11.383723 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:11.383655    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:11.383861 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:11.383777    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:11.383928 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:11.383905    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:11.383980 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:11.383787    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:11.383980 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:11.383971    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:11.384124 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:11.384092    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:11.544272 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:11.544248    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:12.492394 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:12.492165    2570 generic.go:356] "Generic (PLEG): container finished" podID="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" containerID="04b7e5c487388afec471001bac792c6f9d78f95a7241064fe8cde1874e90ac58" exitCode=0
Oct 30 18:46:12.492394 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:12.492248    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ph92c" event={"ID":"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a","Type":"ContainerDied","Data":"04b7e5c487388afec471001bac792c6f9d78f95a7241064fe8cde1874e90ac58"}
Oct 30 18:46:12.497460 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:12.497440    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:12.689132 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:12.689096    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:12.892618 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:12.892588    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:13.084001 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:13.083967    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:13.266846 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:13.266813    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:13.281901 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:13.281872    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:13.282085 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:13.282018    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:46:13.282085 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:13.282051    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:13.282222 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:13.282098    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs podName:4cb022e6-43bd-4d71-9935-f0e91130735b nodeName:}" failed. No retries permitted until 2026-10-30 18:46:29.282080743 +0000 UTC m=+33.507764780 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs") pod "network-metrics-daemon-pgbfm" (UID: "4cb022e6-43bd-4d71-9935-f0e91130735b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:46:13.282222 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:13.282111    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:46:13.282222 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:13.282159    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret podName:b67fee5e-0972-4cfc-987d-b0ae79e6589d nodeName:}" failed. No retries permitted until 2026-10-30 18:46:29.282143678 +0000 UTC m=+33.507827732 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret") pod "global-pull-secret-syncer-wpm6z" (UID: "b67fee5e-0972-4cfc-987d-b0ae79e6589d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:46:13.382780 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:13.382746    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5229t\" (UniqueName: \"kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t\") pod \"network-check-target-42cjm\" (UID: \"ddbd05bb-e311-4a16-af9e-0ab5fa5103c0\") " pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:13.382950 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:13.382927    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Oct 30 18:46:13.383090 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:13.382956    2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Oct 30 18:46:13.383090 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:13.382969    2570 projected.go:196] Error preparing data for projected volume kube-api-access-5229t for pod openshift-network-diagnostics/network-check-target-42cjm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:46:13.383090 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:13.383020    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t podName:ddbd05bb-e311-4a16-af9e-0ab5fa5103c0 nodeName:}" failed. No retries permitted until 2026-10-30 18:46:29.383003841 +0000 UTC m=+33.608687884 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5229t" (UniqueName: "kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t") pod "network-check-target-42cjm" (UID: "ddbd05bb-e311-4a16-af9e-0ab5fa5103c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Oct 30 18:46:13.383215 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:13.383153    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:13.383215 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:13.383207    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:13.383268 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:13.383229    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:13.383325 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:13.383306    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:13.383401 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:13.383382    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:13.383455 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:13.383425    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:13.444961 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:13.444935    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:13.498472 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:13.498421    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp" event={"ID":"652a5a58-ad99-49bb-b371-743c2816482d","Type":"ContainerStarted","Data":"7d16d112b37054c2e0354a34dad15e51b4de9bc55bc59bff9f712f8c3128cfa9"}
Oct 30 18:46:13.629022 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:13.624294    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:14.503341 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:14.503174    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp" event={"ID":"652a5a58-ad99-49bb-b371-743c2816482d","Type":"ContainerStarted","Data":"af8be86ac939914da1a63d7410c87a56c74655c5c7fcc87e178f9e6e15bcd20d"}
Oct 30 18:46:14.503712 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:14.503398    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp" event={"ID":"652a5a58-ad99-49bb-b371-743c2816482d","Type":"ContainerStarted","Data":"1e1cb99e79f3941126e1c8ecd40b4dae1f131d1384b5d6a2fc8a7254796d1846"}
Oct 30 18:46:14.503712 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:14.503447    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp" event={"ID":"652a5a58-ad99-49bb-b371-743c2816482d","Type":"ContainerStarted","Data":"6479b50c15e4c206384625be42a8ede9c9d07f023e72308512b9f9c8c53bcf53"}
Oct 30 18:46:14.503712 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:14.503476    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp" event={"ID":"652a5a58-ad99-49bb-b371-743c2816482d","Type":"ContainerStarted","Data":"eca0c29de06e7c707c6169dcb5ce67491b8822da187a806fdb495e6a27ce6a08"}
Oct 30 18:46:14.503712 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:14.503500    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp" event={"ID":"652a5a58-ad99-49bb-b371-743c2816482d","Type":"ContainerStarted","Data":"e6fd3958417773ca8b45447277f5dc5bb359cbdfdb9d08fe90916379aabb7852"}
Oct 30 18:46:14.739735 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:14.739705    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:14.948005 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:14.947942    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:15.147312 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:15.147230    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:15.347295 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:15.347267    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:15.383590 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:15.383568    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:15.383716 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:15.383599    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:15.383716 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:15.383600    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:15.383716 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:15.383682    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:15.383853 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:15.383730    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:15.383853 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:15.383798    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:15.508157 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:15.508133    2570 generic.go:356] "Generic (PLEG): container finished" podID="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" containerID="8d3a9f398d2d5c1829e4f8bd962541214dd0581dc0222273335381945cb0e43d" exitCode=0
Oct 30 18:46:15.508565 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:15.508160    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ph92c" event={"ID":"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a","Type":"ContainerDied","Data":"8d3a9f398d2d5c1829e4f8bd962541214dd0581dc0222273335381945cb0e43d"}
Oct 30 18:46:15.540853 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:15.540831    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:15.718138 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:15.718111    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:16.430466 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:16.430434    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:16.513202 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:16.513166    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp" event={"ID":"652a5a58-ad99-49bb-b371-743c2816482d","Type":"ContainerStarted","Data":"4f8d0c3abac01e740648f0cba6e3505ffec9c5effdac1d7762b22bf3f074b5d8"}
Oct 30 18:46:16.515875 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:16.515852    2570 generic.go:356] "Generic (PLEG): container finished" podID="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" containerID="a80f5bf5a29d3c9c2b28aa8a585c4ada1b321f1fd6908764cd22f3b516a168ab" exitCode=0
Oct 30 18:46:16.515971 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:16.515899    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ph92c" event={"ID":"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a","Type":"ContainerDied","Data":"a80f5bf5a29d3c9c2b28aa8a585c4ada1b321f1fd6908764cd22f3b516a168ab"}
Oct 30 18:46:16.745000 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:16.744939    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:16.940366 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:16.940338    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:17.147676 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:17.147650    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:17.355813 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:17.355788    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:17.384043 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:17.384014    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:17.384131 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:17.384057    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:17.384131 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:17.384116    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:17.384198 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:17.384159    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:17.384261 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:17.384245    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:17.384318 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:17.384300    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:17.520701 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:17.520678    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ph92c" event={"ID":"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a","Type":"ContainerStarted","Data":"88e8497d13e595f3aa3df90a4ca23e2f07d57051b4d620d08102ced16763253f"}
Oct 30 18:46:17.540988 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:17.540967    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:17.636274 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:17.636236    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ph92c" podStartSLOduration=9.973931158 podStartE2EDuration="21.636224149s" totalImagesPullingTime="11.662292991s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:17.635580567 +0000 UTC m=+21.861264626" watchObservedRunningTime="2026-10-30 18:46:17.636224149 +0000 UTC m=+21.861908209"
Oct 30 18:46:17.716278 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:17.716254    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:17.893878 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:17.893818    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:18.072760 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:18.072729    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:18.536087 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:18.536046    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp" event={"ID":"652a5a58-ad99-49bb-b371-743c2816482d","Type":"ContainerStarted","Data":"cbe8d7730806f21f4b6f54893b78cde58dcf624d79a0c7cc172de268ab48cb8e"}
Oct 30 18:46:18.554928 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:18.554897    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:46:18.555080 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:18.555016    2570 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:46:18.555351 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:18.555331    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:46:18.706084 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:18.706015    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp" podStartSLOduration=7.984023024 podStartE2EDuration="22.706002325s" totalImagesPullingTime="14.721979301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:18.705546923 +0000 UTC m=+22.931230982" watchObservedRunningTime="2026-10-30 18:46:18.706002325 +0000 UTC m=+22.931686417"
Oct 30 18:46:19.383334 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:19.383306    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:19.383334 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:19.383334    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:19.383633 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:19.383404    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:19.383633 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:19.383427    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:19.383633 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:19.383494    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:19.383633 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:19.383577    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:19.481117 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:19.481092    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:19.538648 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:19.538630    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:46:19.539065 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:19.538697    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:46:19.539065 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:19.538737    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:46:19.553764 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:19.553742    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:46:19.553865 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:19.553846    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:46:19.700913 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:19.700857    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:21.383848 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:21.383821    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:21.384241 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:21.383914    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:21.384241 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:21.383985    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:21.384241 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:21.384092    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:21.384241 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:21.384131    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:21.384241 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:21.384185    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:22.103232 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:22.103203    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:22.337279 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:22.337246    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:22.650546 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:22.650515    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:22.834159 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:22.834126    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:23.383378 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:23.383346    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:23.383574 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:23.383477    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:23.383891 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:23.383868    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:23.384204 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:23.384098    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:23.384307 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:23.384226    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:23.384376 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:23.384353    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:23.852027 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:23.852006    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-42cjm"]
Oct 30 18:46:23.852380 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:23.852118    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:23.852380 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:23.852202    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:23.858248 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:23.858224    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wpm6z"]
Oct 30 18:46:23.858470 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:23.858443    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:23.858556 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:23.858530    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:23.859182 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:23.859160    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pgbfm"]
Oct 30 18:46:23.859294 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:23.859259    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:23.859389 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:23.859367    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:24.940211 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:24.940164    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:25.125610 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:25.125542    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:25.318802 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:25.318757    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:25.383467 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:25.383413    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:25.383467 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:25.383439    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:25.383603 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:25.383539    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:25.383603 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:25.383570    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:25.383670 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:25.383623    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:25.383702 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:25.383675    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:25.494360 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:25.494337    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:25.676475 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:25.676421    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:25.895373 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:25.895337    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:26.087220 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:26.087199    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:26.281166 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:26.281138    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:26.473645 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:26.473547    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:26.717255 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:26.717229    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:27.384311 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:27.384281    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:27.384311 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:27.384309    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:27.384311 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:27.384320    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:27.384729 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:27.384401    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pgbfm" podUID="4cb022e6-43bd-4d71-9935-f0e91130735b"
Oct 30 18:46:27.384729 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:27.384514    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-42cjm" podUID="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0"
Oct 30 18:46:27.384729 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:27.384577    2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-wpm6z" podUID="b67fee5e-0972-4cfc-987d-b0ae79e6589d"
Oct 30 18:46:28.444057 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.443844    2570 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeReady"
Oct 30 18:46:28.444384 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.444091    2570 kubelet_node_status.go:477] "Fast updating node status as it just became ready"
Oct 30 18:46:28.562421 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.562395    2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-j88bw"]
Oct 30 18:46:28.564423 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.564409    2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6d2f8"]
Oct 30 18:46:28.564534 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.564519    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.566581 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.566555    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.595220 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.595200    2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-krgch"]
Oct 30 18:46:28.597179 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.597164    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-krgch"
Oct 30 18:46:28.614402 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.614385    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:46:28.614496 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.614483    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vjgl9\""
Oct 30 18:46:28.615737 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.615724    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:46:28.615791 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.615736    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:46:28.628280 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.628263    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j88bw"]
Oct 30 18:46:28.631498 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.631483    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6d2f8"]
Oct 30 18:46:28.660554 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.660532    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-krgch"]
Oct 30 18:46:28.694799 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.694750    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6-cert\") pod \"ingress-canary-krgch\" (UID: \"8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6\") " pod="openshift-ingress-canary/ingress-canary-krgch"
Oct 30 18:46:28.694799 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.694785    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecbb798a-2024-4984-89c1-56ad55912e77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.694923 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.694805    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp6fz\" (UniqueName: \"kubernetes.io/projected/3d07d694-4e2c-401a-aeab-0c9c4f530e87-kube-api-access-bp6fz\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.694923 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.694874    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecbb798a-2024-4984-89c1-56ad55912e77-crio-socket\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.694923 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.694900    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecbb798a-2024-4984-89c1-56ad55912e77-data-volume\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.695090 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.694958    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d07d694-4e2c-401a-aeab-0c9c4f530e87-config-volume\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.695090 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.695002    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d07d694-4e2c-401a-aeab-0c9c4f530e87-tmp-dir\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.695163 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.695089    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2gh2\" (UniqueName: \"kubernetes.io/projected/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6-kube-api-access-z2gh2\") pod \"ingress-canary-krgch\" (UID: \"8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6\") " pod="openshift-ingress-canary/ingress-canary-krgch"
Oct 30 18:46:28.695163 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.695111    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsb2s\" (UniqueName: \"kubernetes.io/projected/ecbb798a-2024-4984-89c1-56ad55912e77-kube-api-access-fsb2s\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.695163 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.695131    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecbb798a-2024-4984-89c1-56ad55912e77-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.695163 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.695147    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d07d694-4e2c-401a-aeab-0c9c4f530e87-metrics-tls\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.714161 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.714144    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:46:28.714394 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.714381    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-vvlg2\""
Oct 30 18:46:28.714462 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.714448    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k86kp\""
Oct 30 18:46:28.715217 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.715204    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:46:28.715281 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.715252    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:46:28.715281 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.715253    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:46:28.715281 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.715267    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:46:28.715397 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.715320    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:46:28.796057 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.796023    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecbb798a-2024-4984-89c1-56ad55912e77-crio-socket\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.796153 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.796061    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecbb798a-2024-4984-89c1-56ad55912e77-data-volume\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.796153 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.796086    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d07d694-4e2c-401a-aeab-0c9c4f530e87-config-volume\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.796284 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.796104    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d07d694-4e2c-401a-aeab-0c9c4f530e87-tmp-dir\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.796543 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.796511    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecbb798a-2024-4984-89c1-56ad55912e77-crio-socket\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.796627 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.796421    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2gh2\" (UniqueName: \"kubernetes.io/projected/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6-kube-api-access-z2gh2\") pod \"ingress-canary-krgch\" (UID: \"8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6\") " pod="openshift-ingress-canary/ingress-canary-krgch"
Oct 30 18:46:28.796683 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.796634    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsb2s\" (UniqueName: \"kubernetes.io/projected/ecbb798a-2024-4984-89c1-56ad55912e77-kube-api-access-fsb2s\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.796683 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.796672    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecbb798a-2024-4984-89c1-56ad55912e77-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.796777 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.796689    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecbb798a-2024-4984-89c1-56ad55912e77-data-volume\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.796777 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.796705    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d07d694-4e2c-401a-aeab-0c9c4f530e87-metrics-tls\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.796777 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.796743    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d07d694-4e2c-401a-aeab-0c9c4f530e87-tmp-dir\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.796921 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.796798    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6-cert\") pod \"ingress-canary-krgch\" (UID: \"8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6\") " pod="openshift-ingress-canary/ingress-canary-krgch"
Oct 30 18:46:28.797746 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.797145    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecbb798a-2024-4984-89c1-56ad55912e77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.797746 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.797269    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp6fz\" (UniqueName: \"kubernetes.io/projected/3d07d694-4e2c-401a-aeab-0c9c4f530e87-kube-api-access-bp6fz\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.797746 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.797290    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d07d694-4e2c-401a-aeab-0c9c4f530e87-config-volume\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.797746 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.797680    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecbb798a-2024-4984-89c1-56ad55912e77-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.801426 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.801398    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecbb798a-2024-4984-89c1-56ad55912e77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.801533 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.801427    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d07d694-4e2c-401a-aeab-0c9c4f530e87-metrics-tls\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.801989 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.801972    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6-cert\") pod \"ingress-canary-krgch\" (UID: \"8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6\") " pod="openshift-ingress-canary/ingress-canary-krgch"
Oct 30 18:46:28.854770 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.854740    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2gh2\" (UniqueName: \"kubernetes.io/projected/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6-kube-api-access-z2gh2\") pod \"ingress-canary-krgch\" (UID: \"8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6\") " pod="openshift-ingress-canary/ingress-canary-krgch"
Oct 30 18:46:28.854857 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.854773    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp6fz\" (UniqueName: \"kubernetes.io/projected/3d07d694-4e2c-401a-aeab-0c9c4f530e87-kube-api-access-bp6fz\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.859856 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.859838    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsb2s\" (UniqueName: \"kubernetes.io/projected/ecbb798a-2024-4984-89c1-56ad55912e77-kube-api-access-fsb2s\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.877790 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.877773    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:28.883651 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.883636    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:46:28.905487 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:28.905454    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-krgch"
Oct 30 18:46:29.300634 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.300609    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:29.300770 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.300663    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:29.300770 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:29.300753    2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Oct 30 18:46:29.300770 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:29.300755    2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:46:29.300868 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:29.300802    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret podName:b67fee5e-0972-4cfc-987d-b0ae79e6589d nodeName:}" failed. No retries permitted until 2026-10-30 18:47:01.300790593 +0000 UTC m=+65.526474632 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret") pod "global-pull-secret-syncer-wpm6z" (UID: "b67fee5e-0972-4cfc-987d-b0ae79e6589d") : object "kube-system"/"original-pull-secret" not registered
Oct 30 18:46:29.300868 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:29.300822    2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs podName:4cb022e6-43bd-4d71-9935-f0e91130735b nodeName:}" failed. No retries permitted until 2026-10-30 18:47:01.300815862 +0000 UTC m=+65.526499900 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs") pod "network-metrics-daemon-pgbfm" (UID: "4cb022e6-43bd-4d71-9935-f0e91130735b") : object "openshift-multus"/"metrics-daemon-secret" not registered
Oct 30 18:46:29.314259 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.314242    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-krgch"]
Oct 30 18:46:29.316488 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.316472    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-j88bw"]
Oct 30 18:46:29.320999 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.320979    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6d2f8"]
Oct 30 18:46:29.322995 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:46:29.322975    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4bdfd9_e0b1_4d6b_a66e_4647cd0c64f6.slice/crio-aad11b046a3a03af247ae3e4b2047008c22032eb02ce4ad94ea40e7b52a05268 WatchSource:0}: Error finding container aad11b046a3a03af247ae3e4b2047008c22032eb02ce4ad94ea40e7b52a05268: Status 404 returned error can't find the container with id aad11b046a3a03af247ae3e4b2047008c22032eb02ce4ad94ea40e7b52a05268
Oct 30 18:46:29.328131 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:46:29.328108    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d07d694_4e2c_401a_aeab_0c9c4f530e87.slice/crio-0e7ff7bd6e351621e118ef488ba19ec4e7a6c27eb2d14ae71d3350f49d7fbbc0 WatchSource:0}: Error finding container 0e7ff7bd6e351621e118ef488ba19ec4e7a6c27eb2d14ae71d3350f49d7fbbc0: Status 404 returned error can't find the container with id 0e7ff7bd6e351621e118ef488ba19ec4e7a6c27eb2d14ae71d3350f49d7fbbc0
Oct 30 18:46:29.328663 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:46:29.328640    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecbb798a_2024_4984_89c1_56ad55912e77.slice/crio-fe08928888a95c73bc82dca6980c49086a5b3bc8aab554f3ea99d38b5f5aefb4 WatchSource:0}: Error finding container fe08928888a95c73bc82dca6980c49086a5b3bc8aab554f3ea99d38b5f5aefb4: Status 404 returned error can't find the container with id fe08928888a95c73bc82dca6980c49086a5b3bc8aab554f3ea99d38b5f5aefb4
Oct 30 18:46:29.383925 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.383903    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:46:29.383925 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.383917    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:46:29.384057 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.383922    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:29.401172 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.401152    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5229t\" (UniqueName: \"kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t\") pod \"network-check-target-42cjm\" (UID: \"ddbd05bb-e311-4a16-af9e-0ab5fa5103c0\") " pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:29.435756 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.435732    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-crxtc\""
Oct 30 18:46:29.435882 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.435783    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:46:29.435882 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.435798    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-q9tbn\""
Oct 30 18:46:29.435882 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.435828    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hgxcg\""
Oct 30 18:46:29.435882 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.435871    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:46:29.436671 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.436658    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:46:29.437059 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.437043    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:46:29.444576 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.444555    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5229t\" (UniqueName: \"kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t\") pod \"network-check-target-42cjm\" (UID: \"ddbd05bb-e311-4a16-af9e-0ab5fa5103c0\") " pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:29.558053 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.557949    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-krgch" event={"ID":"8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6","Type":"ContainerStarted","Data":"aad11b046a3a03af247ae3e4b2047008c22032eb02ce4ad94ea40e7b52a05268"}
Oct 30 18:46:29.559350 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.559324    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6d2f8" event={"ID":"ecbb798a-2024-4984-89c1-56ad55912e77","Type":"ContainerStarted","Data":"3d0a9dc86042f14dc37aca7f67f6990877a81e1bd0669a845b5aa9b3d01e3717"}
Oct 30 18:46:29.559464 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.559370    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6d2f8" event={"ID":"ecbb798a-2024-4984-89c1-56ad55912e77","Type":"ContainerStarted","Data":"fe08928888a95c73bc82dca6980c49086a5b3bc8aab554f3ea99d38b5f5aefb4"}
Oct 30 18:46:29.560324 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.560302    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j88bw" event={"ID":"3d07d694-4e2c-401a-aeab-0c9c4f530e87","Type":"ContainerStarted","Data":"0e7ff7bd6e351621e118ef488ba19ec4e7a6c27eb2d14ae71d3350f49d7fbbc0"}
Oct 30 18:46:29.708366 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:29.708330    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:30.074928 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:30.074901    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-42cjm"]
Oct 30 18:46:30.083266 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:46:30.083244    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddbd05bb_e311_4a16_af9e_0ab5fa5103c0.slice/crio-0143f87b71fb5861a7461820eb8d145a3d7692cf0dc40802b3fbc3efe002ded1 WatchSource:0}: Error finding container 0143f87b71fb5861a7461820eb8d145a3d7692cf0dc40802b3fbc3efe002ded1: Status 404 returned error can't find the container with id 0143f87b71fb5861a7461820eb8d145a3d7692cf0dc40802b3fbc3efe002ded1
Oct 30 18:46:30.566832 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:30.566793    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-42cjm" event={"ID":"ddbd05bb-e311-4a16-af9e-0ab5fa5103c0","Type":"ContainerStarted","Data":"0143f87b71fb5861a7461820eb8d145a3d7692cf0dc40802b3fbc3efe002ded1"}
Oct 30 18:46:31.413517 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:31.413484    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:31.572025 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:31.571990    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6d2f8" event={"ID":"ecbb798a-2024-4984-89c1-56ad55912e77","Type":"ContainerStarted","Data":"785bc4aecda98769a5ce3414a7798cdd9e63a470dce991b793b3f15b854741ca"}
Oct 30 18:46:32.270632 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.270605    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:32.537777 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.537699    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:32.578452 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.578159    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j88bw" event={"ID":"3d07d694-4e2c-401a-aeab-0c9c4f530e87","Type":"ContainerStarted","Data":"ec60fc9280b777c2c993ed5622bbbae1d85177e94999be40f2795c0104e47c10"}
Oct 30 18:46:32.578452 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.578220    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-j88bw" event={"ID":"3d07d694-4e2c-401a-aeab-0c9c4f530e87","Type":"ContainerStarted","Data":"a1224d97d82d1c79813d0a92e6dd676b5538d8f803c30fa9728d1dbba4fe64c8"}
Oct 30 18:46:32.578452 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.578301    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:32.720944 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.720909    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:32.912400 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.912371    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:32.912708 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.912670    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-j88bw" podStartSLOduration=2.553598431 podStartE2EDuration="4.912657431s" totalImagesPullingTime="2.359059s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:32.698425999 +0000 UTC m=+36.924110061" watchObservedRunningTime="2026-10-30 18:46:32.912657431 +0000 UTC m=+37.138341491"
Oct 30 18:46:32.913062 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.913028    2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"]
Oct 30 18:46:32.915309 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.915291    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:32.967567 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.967546    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 18:46:32.967791 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.967774    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:46:32.968089 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.968072    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:46:32.968334 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.968315    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-jd5cp\""
Oct 30 18:46:32.968412 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:32.968360    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 18:46:33.011884 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.011856    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 18:46:33.012909 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.012881    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"]
Oct 30 18:46:33.028089 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.028058    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/42b2d2d6-78f0-457a-b182-09dd65dc3372-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:33.028257 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.028156    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42b2d2d6-78f0-457a-b182-09dd65dc3372-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:33.028257 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.028193    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42b2d2d6-78f0-457a-b182-09dd65dc3372-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:33.028257 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.028234    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtds7\" (UniqueName: \"kubernetes.io/projected/42b2d2d6-78f0-457a-b182-09dd65dc3372-kube-api-access-jtds7\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:33.064812 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.064731    2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"]
Oct 30 18:46:33.067149 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.067127    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.068306 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.068279    2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qdr6m"]
Oct 30 18:46:33.070938 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.070916    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.109205 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.109176    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:33.117374 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.117328    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 18:46:33.117374 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.117332    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-84sx4\""
Oct 30 18:46:33.117374 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.117368    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 18:46:33.121186 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.121167    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 18:46:33.121290 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.121253    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 18:46:33.129159 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.129133    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42b2d2d6-78f0-457a-b182-09dd65dc3372-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:33.130939 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.129175    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42b2d2d6-78f0-457a-b182-09dd65dc3372-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:33.130939 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.129223    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtds7\" (UniqueName: \"kubernetes.io/projected/42b2d2d6-78f0-457a-b182-09dd65dc3372-kube-api-access-jtds7\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:33.130939 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.129269    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/42b2d2d6-78f0-457a-b182-09dd65dc3372-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:33.130939 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.130679    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42b2d2d6-78f0-457a-b182-09dd65dc3372-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:33.134460 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.134439    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/42b2d2d6-78f0-457a-b182-09dd65dc3372-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:33.134559 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.134433    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42b2d2d6-78f0-457a-b182-09dd65dc3372-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:33.156367 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.156342    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"]
Oct 30 18:46:33.189363 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.189331    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtds7\" (UniqueName: \"kubernetes.io/projected/42b2d2d6-78f0-457a-b182-09dd65dc3372-kube-api-access-jtds7\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:33.217998 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.217976    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 18:46:33.218884 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.218860    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jrxvz\""
Oct 30 18:46:33.218884 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.218868    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 18:46:33.229929 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.229897    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfhmw\" (UniqueName: \"kubernetes.io/projected/ca586ba2-2d05-45f8-aab9-127b93ea1c60-kube-api-access-wfhmw\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.230081 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.229936    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.230081 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.229966    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca586ba2-2d05-45f8-aab9-127b93ea1c60-metrics-client-ca\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.230192 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.230101    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e786874-ce56-46f8-87f5-920dacacd9fc-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.230192 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.230173    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqxts\" (UniqueName: \"kubernetes.io/projected/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-api-access-mqxts\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.230271 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.230202    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.230311 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.230295    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-tls\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.230350 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.230328    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-textfile\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.230403 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.230375    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-root\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.230449 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.230427    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0e786874-ce56-46f8-87f5-920dacacd9fc-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.230486 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.230453    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-wtmp\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.230531 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.230485    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.230531 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.230514    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-sys\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.230623 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.230568    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-accelerators-collector-config\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.230663 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.230631    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.230727 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.230713    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:46:33.307588 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.307402    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:33.331260 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331193    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-tls\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.331260 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331245    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-textfile\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.331432 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331388    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-root\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.331480 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331433    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0e786874-ce56-46f8-87f5-920dacacd9fc-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.331480 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331457    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-wtmp\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.331575 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331477    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.331575 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331506    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-sys\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.331575 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331529    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-accelerators-collector-config\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.331575 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331563    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.331727 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331586    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfhmw\" (UniqueName: \"kubernetes.io/projected/ca586ba2-2d05-45f8-aab9-127b93ea1c60-kube-api-access-wfhmw\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.331727 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331587    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-textfile\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.331727 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331604    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.331727 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331620    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca586ba2-2d05-45f8-aab9-127b93ea1c60-metrics-client-ca\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.331727 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331645    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e786874-ce56-46f8-87f5-920dacacd9fc-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.331727 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331671    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqxts\" (UniqueName: \"kubernetes.io/projected/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-api-access-mqxts\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.331727 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331704    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.331953 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331892    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0e786874-ce56-46f8-87f5-920dacacd9fc-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.331996 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.331948    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-root\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.332085 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.332069    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-wtmp\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.332874 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.332849    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-accelerators-collector-config\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.332973 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.332899    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.332973 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.332922    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-sys\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.336844 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.334805    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e786874-ce56-46f8-87f5-920dacacd9fc-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.336844 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.334982    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.336844 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.335093    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca586ba2-2d05-45f8-aab9-127b93ea1c60-metrics-client-ca\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.336844 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.335227    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-tls\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.336844 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.336007    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.340110 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.340087    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.390024 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.390000    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfhmw\" (UniqueName: \"kubernetes.io/projected/ca586ba2-2d05-45f8-aab9-127b93ea1c60-kube-api-access-wfhmw\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.390128 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.390104    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:46:33.392434 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.392411    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqxts\" (UniqueName: \"kubernetes.io/projected/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-api-access-mqxts\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.401205 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:46:33.401177    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca586ba2_2d05_45f8_aab9_127b93ea1c60.slice/crio-80ff3b20e76a7e6edd5a96faa8e3e9373e5b4fd8d417efa5e26120fc44e52d74 WatchSource:0}: Error finding container 80ff3b20e76a7e6edd5a96faa8e3e9373e5b4fd8d417efa5e26120fc44e52d74: Status 404 returned error can't find the container with id 80ff3b20e76a7e6edd5a96faa8e3e9373e5b4fd8d417efa5e26120fc44e52d74
Oct 30 18:46:33.537634 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.537602    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:33.583095 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.582980    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6d2f8" event={"ID":"ecbb798a-2024-4984-89c1-56ad55912e77","Type":"ContainerStarted","Data":"c8bf50d3647c2ade9e7591130f233af3dc8b3cd5c13450921a00809b7ad5710b"}
Oct 30 18:46:33.584795 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.584768    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-krgch" event={"ID":"8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6","Type":"ContainerStarted","Data":"765d10b1ec91a0c5346f4ec6688c13c6b34bb56ef78315bed7948d8427c9d0e8"}
Oct 30 18:46:33.586145 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.586120    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qdr6m" event={"ID":"ca586ba2-2d05-45f8-aab9-127b93ea1c60","Type":"ContainerStarted","Data":"80ff3b20e76a7e6edd5a96faa8e3e9373e5b4fd8d417efa5e26120fc44e52d74"}
Oct 30 18:46:33.662932 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.662904    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"]
Oct 30 18:46:33.672435 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:46:33.672412    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42b2d2d6_78f0_457a_b182_09dd65dc3372.slice/crio-f2f6cee433b8f8ac45f4eaedafa95e02bc0cc4cfb7d070615acc3657d05f8d62 WatchSource:0}: Error finding container f2f6cee433b8f8ac45f4eaedafa95e02bc0cc4cfb7d070615acc3657d05f8d62: Status 404 returned error can't find the container with id f2f6cee433b8f8ac45f4eaedafa95e02bc0cc4cfb7d070615acc3657d05f8d62
Oct 30 18:46:33.684219 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.684198    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:46:33.696170 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.696125    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6d2f8" podStartSLOduration=2.3796166149999998 podStartE2EDuration="5.696026838s" totalImagesPullingTime="3.316410223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:46:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:33.695241554 +0000 UTC m=+37.920925618" watchObservedRunningTime="2026-10-30 18:46:33.696026838 +0000 UTC m=+37.921710898"
Oct 30 18:46:33.809240 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:33.809197    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-krgch" podStartSLOduration=2.210975535 podStartE2EDuration="5.809181395s" totalImagesPullingTime="3.59820586s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:33.808630352 +0000 UTC m=+38.034314411" watchObservedRunningTime="2026-10-30 18:46:33.809181395 +0000 UTC m=+38.034865455"
Oct 30 18:46:34.048294 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:34.048263    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:34.070301 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:34.070278    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"]
Oct 30 18:46:34.079098 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:46:34.079071    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e786874_ce56_46f8_87f5_920dacacd9fc.slice/crio-8dafede46b3521ea090abb5b0da5a438bed9a6677568536d854d1baed27fec81 WatchSource:0}: Error finding container 8dafede46b3521ea090abb5b0da5a438bed9a6677568536d854d1baed27fec81: Status 404 returned error can't find the container with id 8dafede46b3521ea090abb5b0da5a438bed9a6677568536d854d1baed27fec81
Oct 30 18:46:34.430677 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:34.430647    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:34.590404 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:34.590370    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-42cjm" event={"ID":"ddbd05bb-e311-4a16-af9e-0ab5fa5103c0","Type":"ContainerStarted","Data":"3c50a8a07b1549fb1dae246efbca9d968637dfae39afabe93c99a3308a3032c6"}
Oct 30 18:46:34.590866 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:34.590491    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:46:34.591797 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:34.591770    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl" event={"ID":"0e786874-ce56-46f8-87f5-920dacacd9fc","Type":"ContainerStarted","Data":"8dafede46b3521ea090abb5b0da5a438bed9a6677568536d854d1baed27fec81"}
Oct 30 18:46:34.593932 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:34.593817    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc" event={"ID":"42b2d2d6-78f0-457a-b182-09dd65dc3372","Type":"ContainerStarted","Data":"ea5db45d2b188053be535f0c997fafa40a41134c9c5e997e40bc2f45506a00b3"}
Oct 30 18:46:34.593932 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:34.593877    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc" event={"ID":"42b2d2d6-78f0-457a-b182-09dd65dc3372","Type":"ContainerStarted","Data":"f2cc74ea939b2c612241b3705b9a0451d4aa16f4c3dbda4fc2dccbc57a63693a"}
Oct 30 18:46:34.593932 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:34.593906    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc" event={"ID":"42b2d2d6-78f0-457a-b182-09dd65dc3372","Type":"ContainerStarted","Data":"f2f6cee433b8f8ac45f4eaedafa95e02bc0cc4cfb7d070615acc3657d05f8d62"}
Oct 30 18:46:34.700108 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:34.699997    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-42cjm" podStartSLOduration=34.888471116 podStartE2EDuration="38.699985848s" totalImagesPullingTime="3.811514732s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:34.699373609 +0000 UTC m=+38.925057682" watchObservedRunningTime="2026-10-30 18:46:34.699985848 +0000 UTC m=+38.925669908"
Oct 30 18:46:35.060828 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.060651    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:35.266131 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.266100    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:35.275280 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.273491    2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:46:35.276841 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.276816    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.329715 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.329691    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 18:46:35.329835 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.329768    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 18:46:35.374540 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.374424    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 18:46:35.374540 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.374489    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 18:46:35.374540 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.374491    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 18:46:35.374746 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.374709    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 18:46:35.374746 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.374716    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 18:46:35.374837 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.374753    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 18:46:35.420988 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.420935    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 18:46:35.422409 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.422382    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:46:35.429297 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.429278    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mgrlf\""
Oct 30 18:46:35.447731 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.447707    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-config-out\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.447847 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.447752    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.447847 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.447787    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.447921 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.447874    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.447951 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.447920    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.447951 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.447942    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.448009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.447960    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-web-config\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.448009 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.447990    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.448106 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.448015    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8brjr\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-kube-api-access-8brjr\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.448106 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.448058    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-config-volume\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.448106 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.448094    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.448269 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.448121    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.448269 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.448139    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.548840 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.548751    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-config-out\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.548840 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.548799    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.549064 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.548840    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.549064 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.548868    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.549064 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.548902    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.549064 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.548925    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.549064 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.548954    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-web-config\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.549064 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.549002    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.549064 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.549028    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8brjr\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-kube-api-access-8brjr\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.549385 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.549073    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-config-volume\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.549385 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.549108    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.549385 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.549146    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.549385 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.549176    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.550525 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.549910    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.550525 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.550204    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.550525 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.550321    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.554689 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.554662    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.554817 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.554685    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.554817 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.554685    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.554817 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.554722    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.554817 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.554752    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.555091 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.555018    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-web-config\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.555596 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.555571    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-config-volume\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.555934 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.555912    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.558538 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.558519    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-config-out\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.600449 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.600416    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc" event={"ID":"42b2d2d6-78f0-457a-b182-09dd65dc3372","Type":"ContainerStarted","Data":"c5cc73b13bdd9b3a6e401de84fd37582e2043e5ea910b6a20a9042d40f510704"}
Oct 30 18:46:35.602357 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.602334    2570 generic.go:356] "Generic (PLEG): container finished" podID="ca586ba2-2d05-45f8-aab9-127b93ea1c60" containerID="78104c5c61a780315a5419e8b4852b5182dcbb7f7960dd9dc3fedae7dcde499e" exitCode=0
Oct 30 18:46:35.602444 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.602421    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qdr6m" event={"ID":"ca586ba2-2d05-45f8-aab9-127b93ea1c60","Type":"ContainerDied","Data":"78104c5c61a780315a5419e8b4852b5182dcbb7f7960dd9dc3fedae7dcde499e"}
Oct 30 18:46:35.621391 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.621373    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8brjr\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-kube-api-access-8brjr\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.707706 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.707685    2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-5d769f5588-7flhp"]
Oct 30 18:46:35.710439 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.710411    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.760393 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.760358    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 18:46:35.760393 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.760376    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 18:46:35.760393 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.760388    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-kc6rd\""
Oct 30 18:46:35.760602 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.760442    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2kppepquv6fir\""
Oct 30 18:46:35.760602 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.760458    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 18:46:35.760602 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.760467    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 18:46:35.760931 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.760916    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 18:46:35.774122 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.774087    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc" podStartSLOduration=2.2559651 podStartE2EDuration="3.774074114s" totalImagesPullingTime="1.518109014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:35.71680254 +0000 UTC m=+39.942486599" watchObservedRunningTime="2026-10-30 18:46:35.774074114 +0000 UTC m=+39.999758174"
Oct 30 18:46:35.774795 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.774782    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d769f5588-7flhp"]
Oct 30 18:46:35.851766 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.851742    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ae72d3a-8b68-4d30-93d7-f03418559928-tmp\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.851864 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.851777    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-grpc-tls\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.851864 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.851798    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2t26\" (UniqueName: \"kubernetes.io/projected/9ae72d3a-8b68-4d30-93d7-f03418559928-kube-api-access-z2t26\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.851978 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.851878    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.851978 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.851962    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.852074 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.851993    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ae72d3a-8b68-4d30-93d7-f03418559928-metrics-client-ca\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.852133 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.852077    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.852133 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.852105    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-tls\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.852226 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.852130    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.900310 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.900285    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:46:35.955306 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.953208    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.955306 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.953787    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.955306 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.953813    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ae72d3a-8b68-4d30-93d7-f03418559928-metrics-client-ca\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.955306 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.954656    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ae72d3a-8b68-4d30-93d7-f03418559928-metrics-client-ca\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.955306 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.954737    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.955306 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.954807    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-tls\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.955306 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.955167    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.955667 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.955586    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ae72d3a-8b68-4d30-93d7-f03418559928-tmp\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.955980 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.955959    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ae72d3a-8b68-4d30-93d7-f03418559928-tmp\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.956162 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.956143    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-grpc-tls\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.956574 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.956553    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2t26\" (UniqueName: \"kubernetes.io/projected/9ae72d3a-8b68-4d30-93d7-f03418559928-kube-api-access-z2t26\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.956942 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.956920    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.958277 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.957772    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.958277 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.958229    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.959290 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.959265    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-tls\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.959509 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.959488    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-grpc-tls\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:35.960818 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:35.960463    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:36.014703 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.014677    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2t26\" (UniqueName: \"kubernetes.io/projected/9ae72d3a-8b68-4d30-93d7-f03418559928-kube-api-access-z2t26\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:36.030530 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.030507    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:36.112646 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.112620    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:36.295890 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.295864    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:36.336298 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.336228    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:46:36.343319 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:46:36.343281    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2717f98e_b200_43c5_a089_73fb6a522933.slice/crio-a5dcaf13f72dae8ace0899f1b37e8f9af530604a73d38e414d2bdf39ba069334 WatchSource:0}: Error finding container a5dcaf13f72dae8ace0899f1b37e8f9af530604a73d38e414d2bdf39ba069334: Status 404 returned error can't find the container with id a5dcaf13f72dae8ace0899f1b37e8f9af530604a73d38e414d2bdf39ba069334
Oct 30 18:46:36.432097 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.432064    2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-77b7f76bd4-6rfpv"]
Oct 30 18:46:36.435950 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.435760    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-5d769f5588-7flhp"]
Oct 30 18:46:36.435950 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.435918    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.486812 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.486789    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lk2tt\""
Oct 30 18:46:36.486903 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.486875    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 18:46:36.486984 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.486947    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 18:46:36.487021 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.487012    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 18:46:36.487081 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.487017    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 18:46:36.487132 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.487118    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 18:46:36.487165 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.487123    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 18:46:36.487193 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.487179    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 18:46:36.488167 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.488151    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 18:46:36.496402 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.496384    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b7f76bd4-6rfpv"]
Oct 30 18:46:36.562863 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.562836    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.563010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.562872    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.563010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.562905    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.563010 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.562926    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.563221 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.563062    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.563221 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.563156    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f359854-4031-4cf7-9ef4-877e4a90d249-tmp\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.563221 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.563198    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vsptj\" (UniqueName: \"kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.563344 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.563223    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.607363 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.607284    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl" event={"ID":"0e786874-ce56-46f8-87f5-920dacacd9fc","Type":"ContainerStarted","Data":"288b8c1a35eaf9875d9f07394aca52e7ff9f9fbb47ad1e1f623168b713d2bb09"}
Oct 30 18:46:36.607909 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.607378    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl" event={"ID":"0e786874-ce56-46f8-87f5-920dacacd9fc","Type":"ContainerStarted","Data":"28ab2e901cccdafeeae85980b50ad6b63a6a3b7a6413ecaa97db075808bb09b5"}
Oct 30 18:46:36.607909 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.607408    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl" event={"ID":"0e786874-ce56-46f8-87f5-920dacacd9fc","Type":"ContainerStarted","Data":"31bac492f3d9d1f64c26eda9448bed7431b46b720300632bb19ffb752f141e4a"}
Oct 30 18:46:36.609530 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.609506    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qdr6m" event={"ID":"ca586ba2-2d05-45f8-aab9-127b93ea1c60","Type":"ContainerStarted","Data":"3f54b004c79449b01f06666e3bea058f746c286c0a97de30590d994bd32a60d5"}
Oct 30 18:46:36.609624 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.609545    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qdr6m" event={"ID":"ca586ba2-2d05-45f8-aab9-127b93ea1c60","Type":"ContainerStarted","Data":"51063f93e7cf1a8caf0461ad80d8f266c4960a31a04cf9eac0ad76a48ef57558"}
Oct 30 18:46:36.610706 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.610685    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp" event={"ID":"9ae72d3a-8b68-4d30-93d7-f03418559928","Type":"ContainerStarted","Data":"1b28ccf8718b1b23df3e6129770c007aef7bbf94cb020f329fa3135fbe247c8d"}
Oct 30 18:46:36.611871 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.611852    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerStarted","Data":"a5dcaf13f72dae8ace0899f1b37e8f9af530604a73d38e414d2bdf39ba069334"}
Oct 30 18:46:36.664467 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.664444    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.664566 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.664490    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f359854-4031-4cf7-9ef4-877e4a90d249-tmp\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.664627 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.664606    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsptj\" (UniqueName: \"kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.664676 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.664637    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.664676 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.664666    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.664779 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.664685    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.664871 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.664855    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f359854-4031-4cf7-9ef4-877e4a90d249-tmp\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.664929 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.664897    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.664929 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.664922    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.665396 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.665374    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.665490 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.665396    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.665569 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.665545    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.665851 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.665833    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.668391 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.667938    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.668391 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.668104    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.722853 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.722828    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vsptj\" (UniqueName: \"kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:36.725999 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.725962    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl" podStartSLOduration=2.053622656 podStartE2EDuration="3.725950174s" totalImagesPullingTime="1.672327518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:36.725397666 +0000 UTC m=+40.951081727" watchObservedRunningTime="2026-10-30 18:46:36.725950174 +0000 UTC m=+40.951634234"
Oct 30 18:46:36.751015 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:36.750989    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:37.118886 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.118836    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qdr6m" podStartSLOduration=2.563119777 podStartE2EDuration="4.118821226s" totalImagesPullingTime="1.479543631s" totalInitContainerRuntime="76.157818ms" isStatefulPod=true podCreationTimestamp="2026-10-30 18:46:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:36.838553348 +0000 UTC m=+41.064237420" watchObservedRunningTime="2026-10-30 18:46:37.118821226 +0000 UTC m=+41.344505280"
Oct 30 18:46:37.119647 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.119629    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77b7f76bd4-6rfpv"]
Oct 30 18:46:37.127174 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:46:37.127148    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f359854_4031_4cf7_9ef4_877e4a90d249.slice/crio-7bc50e16469aee5af6b8dab32fa3c80a8f1923d1eb0f3763e875fc6c42bec2b8 WatchSource:0}: Error finding container 7bc50e16469aee5af6b8dab32fa3c80a8f1923d1eb0f3763e875fc6c42bec2b8: Status 404 returned error can't find the container with id 7bc50e16469aee5af6b8dab32fa3c80a8f1923d1eb0f3763e875fc6c42bec2b8
Oct 30 18:46:37.211849 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.211790    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:37.465988 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.465909    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:37.617349 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.617311    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b7f76bd4-6rfpv" event={"ID":"1f359854-4031-4cf7-9ef4-877e4a90d249","Type":"ContainerStarted","Data":"7bc50e16469aee5af6b8dab32fa3c80a8f1923d1eb0f3763e875fc6c42bec2b8"}
Oct 30 18:46:37.647297 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.647210    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:37.681406 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.681376    2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"]
Oct 30 18:46:37.684261 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.684238    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:46:37.733411 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.733335    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 18:46:37.733411 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.733348    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-n4vtv\""
Oct 30 18:46:37.743800 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.743774    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"]
Oct 30 18:46:37.776649 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.776622    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btbrv\" (UniqueName: \"kubernetes.io/projected/3acfed5c-f071-4450-9e72-d923cc492d3f-kube-api-access-btbrv\") pod \"monitoring-plugin-77f4f76594-fnkq7\" (UID: \"3acfed5c-f071-4450-9e72-d923cc492d3f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:46:37.776842 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.776824    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3acfed5c-f071-4450-9e72-d923cc492d3f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-fnkq7\" (UID: \"3acfed5c-f071-4450-9e72-d923cc492d3f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:46:37.826726 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.826703    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:37.877457 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.877310    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btbrv\" (UniqueName: \"kubernetes.io/projected/3acfed5c-f071-4450-9e72-d923cc492d3f-kube-api-access-btbrv\") pod \"monitoring-plugin-77f4f76594-fnkq7\" (UID: \"3acfed5c-f071-4450-9e72-d923cc492d3f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:46:37.877633 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.877521    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3acfed5c-f071-4450-9e72-d923cc492d3f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-fnkq7\" (UID: \"3acfed5c-f071-4450-9e72-d923cc492d3f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:46:37.880758 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.880738    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3acfed5c-f071-4450-9e72-d923cc492d3f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-fnkq7\" (UID: \"3acfed5c-f071-4450-9e72-d923cc492d3f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:46:37.935451 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.935423    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btbrv\" (UniqueName: \"kubernetes.io/projected/3acfed5c-f071-4450-9e72-d923cc492d3f-kube-api-access-btbrv\") pod \"monitoring-plugin-77f4f76594-fnkq7\" (UID: \"3acfed5c-f071-4450-9e72-d923cc492d3f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:46:37.998174 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:37.998144    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:46:38.006978 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.006952    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:38.090654 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.090627    2570 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"]
Oct 30 18:46:38.094307 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.093902    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.143326 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.143300    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 18:46:38.143477 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.143332    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 18:46:38.143477 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.143331    2570 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 18:46:38.143578 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.143524    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-uf0a4mc8vfn5\""
Oct 30 18:46:38.143578 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.143547    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qmmm4\""
Oct 30 18:46:38.143666 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.143584    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 18:46:38.154151 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.154127    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"]
Oct 30 18:46:38.180092 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.180064    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-secret-metrics-server-tls\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.180234 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.180102    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-secret-metrics-server-client-certs\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.180234 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.180205    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-client-ca-bundle\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.180337 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.180235    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9cjq\" (UniqueName: \"kubernetes.io/projected/263a55c9-0e5a-47bc-bec1-20cd01e12942-kube-api-access-t9cjq\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.180337 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.180272    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/263a55c9-0e5a-47bc-bec1-20cd01e12942-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.180337 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.180297    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/263a55c9-0e5a-47bc-bec1-20cd01e12942-metrics-server-audit-profiles\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.180337 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.180325    2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/263a55c9-0e5a-47bc-bec1-20cd01e12942-audit-log\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.281238 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.281158    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-client-ca-bundle\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.281238 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.281198    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9cjq\" (UniqueName: \"kubernetes.io/projected/263a55c9-0e5a-47bc-bec1-20cd01e12942-kube-api-access-t9cjq\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.281448 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.281318    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/263a55c9-0e5a-47bc-bec1-20cd01e12942-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.281448 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.281358    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/263a55c9-0e5a-47bc-bec1-20cd01e12942-metrics-server-audit-profiles\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.281448 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.281391    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/263a55c9-0e5a-47bc-bec1-20cd01e12942-audit-log\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.281579 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.281454    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-secret-metrics-server-tls\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.281579 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.281502    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-secret-metrics-server-client-certs\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.281967 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.281943    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/263a55c9-0e5a-47bc-bec1-20cd01e12942-audit-log\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.282083 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.281955    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/263a55c9-0e5a-47bc-bec1-20cd01e12942-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.282319 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.282273    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/263a55c9-0e5a-47bc-bec1-20cd01e12942-metrics-server-audit-profiles\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.284143 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.284115    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-client-ca-bundle\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.284276 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.284252    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-secret-metrics-server-client-certs\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.284367 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.284350    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-secret-metrics-server-tls\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.343236 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.343210    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9cjq\" (UniqueName: \"kubernetes.io/projected/263a55c9-0e5a-47bc-bec1-20cd01e12942-kube-api-access-t9cjq\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.382227 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.382196    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"]
Oct 30 18:46:38.390074 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:46:38.390046    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3acfed5c_f071_4450_9e72_d923cc492d3f.slice/crio-6b05236719987d7322ada67217d827746076b2352df5d72bb8da7b6b5135b9fa WatchSource:0}: Error finding container 6b05236719987d7322ada67217d827746076b2352df5d72bb8da7b6b5135b9fa: Status 404 returned error can't find the container with id 6b05236719987d7322ada67217d827746076b2352df5d72bb8da7b6b5135b9fa
Oct 30 18:46:38.408229 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.408210    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:38.624346 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.624255    2570 generic.go:356] "Generic (PLEG): container finished" podID="2717f98e-b200-43c5-a089-73fb6a522933" containerID="070fcf583e97a5546d1b72cba3246e572a54913e0c473fbbb39dd8e861eaa71f" exitCode=0
Oct 30 18:46:38.625139 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.624348    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerDied","Data":"070fcf583e97a5546d1b72cba3246e572a54913e0c473fbbb39dd8e861eaa71f"}
Oct 30 18:46:38.625984 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.625954    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7" event={"ID":"3acfed5c-f071-4450-9e72-d923cc492d3f","Type":"ContainerStarted","Data":"6b05236719987d7322ada67217d827746076b2352df5d72bb8da7b6b5135b9fa"}
Oct 30 18:46:38.835616 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:38.835579    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"]
Oct 30 18:46:38.844017 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:46:38.843972    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod263a55c9_0e5a_47bc_bec1_20cd01e12942.slice/crio-fc2294e8c03f9d3b1327c54fdfcc69e498a708099b0009121cf32543672797ae WatchSource:0}: Error finding container fc2294e8c03f9d3b1327c54fdfcc69e498a708099b0009121cf32543672797ae: Status 404 returned error can't find the container with id fc2294e8c03f9d3b1327c54fdfcc69e498a708099b0009121cf32543672797ae
Oct 30 18:46:39.634127 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:39.633850    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp" event={"ID":"9ae72d3a-8b68-4d30-93d7-f03418559928","Type":"ContainerStarted","Data":"8163086f2d1f9365af1b44071d49560ef4e58a55cd9a57557d6d7ca8ecc4f8d9"}
Oct 30 18:46:39.634590 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:39.634164    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp" event={"ID":"9ae72d3a-8b68-4d30-93d7-f03418559928","Type":"ContainerStarted","Data":"c29c0de7b1d594e89fc79e04ccfc1fa90158025e68986d9c4d005420870f8b4e"}
Oct 30 18:46:39.634590 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:39.634212    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp" event={"ID":"9ae72d3a-8b68-4d30-93d7-f03418559928","Type":"ContainerStarted","Data":"554f377495650666bd0f71d73109afa598c5978c8c77d1335319e598e139b98a"}
Oct 30 18:46:39.635664 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:39.635608    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw" event={"ID":"263a55c9-0e5a-47bc-bec1-20cd01e12942","Type":"ContainerStarted","Data":"fc2294e8c03f9d3b1327c54fdfcc69e498a708099b0009121cf32543672797ae"}
Oct 30 18:46:40.300918 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:40.300881    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:40.475889 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:40.475866    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:40.654105 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:40.653481    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp" event={"ID":"9ae72d3a-8b68-4d30-93d7-f03418559928","Type":"ContainerStarted","Data":"34f2dcc9911134116aede6be32c5cdb5acc984b63fc7059fa380bc4835475c64"}
Oct 30 18:46:40.654105 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:40.653606    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp" event={"ID":"9ae72d3a-8b68-4d30-93d7-f03418559928","Type":"ContainerStarted","Data":"a19be5d80a134b81d133c00c16745e242aa14f42b82a974c08efac7a6f74422f"}
Oct 30 18:46:40.660335 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:40.660312    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7" event={"ID":"3acfed5c-f071-4450-9e72-d923cc492d3f","Type":"ContainerStarted","Data":"fe6fbf161a3a06bb4a5c0df9ed787ed5f1119736752f3c6538329afbec490ba1"}
Oct 30 18:46:40.661596 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:40.661561    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:46:40.668493 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:40.668476    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:46:40.774914 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:40.774852    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7" podStartSLOduration=1.787020226 podStartE2EDuration="3.774833222s" totalImagesPullingTime="1.987812996s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:40.773224571 +0000 UTC m=+44.998908625" watchObservedRunningTime="2026-10-30 18:46:40.774833222 +0000 UTC m=+45.000517285"
Oct 30 18:46:41.666941 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:41.666860    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp" event={"ID":"9ae72d3a-8b68-4d30-93d7-f03418559928","Type":"ContainerStarted","Data":"e8df9a6a3909ef00eed9c459b731aeec94cb12cbbaabb504639e1dc79d5c3435"}
Oct 30 18:46:41.667397 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:41.667004    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:41.670255 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:41.670228    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerStarted","Data":"4c8e2d8371de748318149aeb5481965ee81592f199b46b0e6c6178107ef26634"}
Oct 30 18:46:41.670382 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:41.670299    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerStarted","Data":"c6f9238b7c08c2ba14f3ad3e2b6f1e5a9dab86fd86eda1100422a47632ad251c"}
Oct 30 18:46:41.670382 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:41.670341    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerStarted","Data":"00bb712bc3823d540d10530c13b97af9a11ea6a844629276c63e4aa31ca2223e"}
Oct 30 18:46:41.670382 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:41.670379    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerStarted","Data":"6a71f155d5d440b454c54c6f8a1bb50c234627d55602ed3bd9dc1277ac792466"}
Oct 30 18:46:41.670495 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:41.670416    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerStarted","Data":"7b4e8406dfb3b12b172d10289225b3d68a882a76e59a72e174d9f7af228969d5"}
Oct 30 18:46:41.670495 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:41.670456    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerStarted","Data":"029c727e72eaefa3b5c36c5839eb42561d83c1751195646b786093e959132bc8"}
Oct 30 18:46:41.671872 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:41.671847    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b7f76bd4-6rfpv" event={"ID":"1f359854-4031-4cf7-9ef4-877e4a90d249","Type":"ContainerStarted","Data":"7522e19ea9d180138546b0d27e32ca0e83b22b254fe812a9c2916579fbe3b480"}
Oct 30 18:46:41.673419 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:41.673396    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw" event={"ID":"263a55c9-0e5a-47bc-bec1-20cd01e12942","Type":"ContainerStarted","Data":"196e1f3e6d0b34fe1ee50407588b0268c29adb0e019ce642af533ce8e39923f0"}
Oct 30 18:46:41.786200 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:41.786152    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp" podStartSLOduration=2.9728684100000002 podStartE2EDuration="6.786139483s" totalImagesPullingTime="3.813271073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:41.784914741 +0000 UTC m=+46.010598800" watchObservedRunningTime="2026-10-30 18:46:41.786139483 +0000 UTC m=+46.011823542"
Oct 30 18:46:41.898002 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:41.897956    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-77b7f76bd4-6rfpv" podStartSLOduration=1.740356345 podStartE2EDuration="5.89794176s" totalImagesPullingTime="4.157585415s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:41.897323564 +0000 UTC m=+46.123007621" watchObservedRunningTime="2026-10-30 18:46:41.89794176 +0000 UTC m=+46.123625819"
Oct 30 18:46:42.010686 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:42.010629    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw" podStartSLOduration=1.571829137 podStartE2EDuration="4.010611117s" totalImagesPullingTime="2.43878198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:42.009988796 +0000 UTC m=+46.235672857" watchObservedRunningTime="2026-10-30 18:46:42.010611117 +0000 UTC m=+46.236295175"
Oct 30 18:46:42.137123 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:42.137095    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:42.142249 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:42.142211    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.182525239 podStartE2EDuration="7.142198866s" totalImagesPullingTime="3.942531554s" totalInitContainerRuntime="17.142073ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:42.140180382 +0000 UTC m=+46.365864442" watchObservedRunningTime="2026-10-30 18:46:42.142198866 +0000 UTC m=+46.367882926"
Oct 30 18:46:42.323707 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:42.323632    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:42.588821 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:42.588745    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-j88bw"
Oct 30 18:46:43.513832 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:43.513802    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:43.694412 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:43.694373    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:45.212740 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:45.212706    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:45.386315 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:45.386287    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:45.446350 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:45.446325    2570 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b7f76bd4-6rfpv"]
Oct 30 18:46:45.569994 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:45.569961    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:46.296116 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:46.296086    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:46.465802 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:46.465771    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:46.751962 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:46.751942    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:46:47.254438 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:47.254404    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:47.435527 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:47.435499    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:47.682695 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:47.682631    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:46:47.786239 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:47.786195    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:47.967683 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:47.967603    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:48.156061 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:48.155999    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:51.552209 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:51.552182    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:46:52.735568 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:52.735520    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:52.920202 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:52.920169    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:53.295840 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:53.295811    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:53.474441 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:53.474409    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:54.104539 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:54.104508    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:54.355672 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:54.355597    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:55.301913 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:55.301874    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:55.495226 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:55.495193    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:58.244691 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:58.244659    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:58.409296 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:58.409274    2570 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:58.409442 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:58.409321    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:46:58.422192 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:46:58.422146    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:46:58.910547 ip-10-0-139-33 kubenswrapper[2570]: E1030 18:46:58.910510    2570 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:47:01.368215 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:01.368186    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:47:01.368660 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:01.368230    2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:47:01.462470 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:01.462443    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:47:01.462470 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:01.462457    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:47:01.471107 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:01.471084    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:47:01.471220 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:01.471110    2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:47:01.543321 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:01.543301    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-q9tbn\""
Oct 30 18:47:01.546418 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:01.546392    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:47:01.550713 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:01.550695    2570 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hgxcg\""
Oct 30 18:47:01.553827 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:01.553808    2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:47:01.918434 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:01.918371    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-wpm6z"]
Oct 30 18:47:01.928785 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:47:01.926867    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb67fee5e_0972_4cfc_987d_b0ae79e6589d.slice/crio-f8e3edc927fdc1674f512ef088d22ea4aebb6823380e08537e85cf60933eec98 WatchSource:0}: Error finding container f8e3edc927fdc1674f512ef088d22ea4aebb6823380e08537e85cf60933eec98: Status 404 returned error can't find the container with id f8e3edc927fdc1674f512ef088d22ea4aebb6823380e08537e85cf60933eec98
Oct 30 18:47:01.976354 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:01.976281    2570 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pgbfm"]
Oct 30 18:47:01.983611 ip-10-0-139-33 kubenswrapper[2570]: W1030 18:47:01.983587    2570 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb022e6_43bd_4d71_9935_f0e91130735b.slice/crio-3725fc11d97fb1eeafa752dfadf0018bef6aeedbde6ca23e8d8b731fb7f0f2d4 WatchSource:0}: Error finding container 3725fc11d97fb1eeafa752dfadf0018bef6aeedbde6ca23e8d8b731fb7f0f2d4: Status 404 returned error can't find the container with id 3725fc11d97fb1eeafa752dfadf0018bef6aeedbde6ca23e8d8b731fb7f0f2d4
Oct 30 18:47:02.747979 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:02.747947    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-wpm6z" event={"ID":"b67fee5e-0972-4cfc-987d-b0ae79e6589d","Type":"ContainerStarted","Data":"f8e3edc927fdc1674f512ef088d22ea4aebb6823380e08537e85cf60933eec98"}
Oct 30 18:47:02.749085 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:02.749061    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pgbfm" event={"ID":"4cb022e6-43bd-4d71-9935-f0e91130735b","Type":"ContainerStarted","Data":"3725fc11d97fb1eeafa752dfadf0018bef6aeedbde6ca23e8d8b731fb7f0f2d4"}
Oct 30 18:47:03.754186 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:03.754159    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pgbfm" event={"ID":"4cb022e6-43bd-4d71-9935-f0e91130735b","Type":"ContainerStarted","Data":"84e945fc359b0487db5af6e409c51f37c7184059401e754275cafb8cb860932f"}
Oct 30 18:47:04.118909 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:04.118879    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:04.326877 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:04.326844    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:04.760656 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:04.760621    2570 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pgbfm" event={"ID":"4cb022e6-43bd-4d71-9935-f0e91130735b","Type":"ContainerStarted","Data":"7f658e6c9e1c79e7444f1354c3f21c5f9e7f0828d438b0952efce10e5b2336e0"}
Oct 30 18:47:04.874055 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:04.873987    2570 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pgbfm" podStartSLOduration=67.314879064 podStartE2EDuration="1m8.8739726s" totalImagesPullingTime="1.559093536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:04.871319984 +0000 UTC m=+69.097004044" watchObservedRunningTime="2026-10-30 18:47:04.8739726 +0000 UTC m=+69.099656679"
Oct 30 18:47:05.604959 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:05.604928    2570 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:47:05.833402 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:05.833369    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:06.008064 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:06.008019    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:06.191562 ip-10-0-139-33 kubenswrapper[2570]: I1030 18:47:06.191526    2570 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:07.001105 ip-10-0-139-33 systemd[1]: Stopping Kubernetes Kubelet...
Oct 30 18:47:07.013473 ip-10-0-139-33 systemd[1]: kubelet.service: Deactivated successfully.
Oct 30 18:47:07.013632 ip-10-0-139-33 systemd[1]: Stopped Kubernetes Kubelet.
Oct 30 18:47:07.013806 ip-10-0-139-33 systemd[1]: kubelet.service: Consumed 4.251s CPU time.
Oct 30 18:47:07.022761 ip-10-0-139-33 systemd[1]: Starting Kubernetes Kubelet...
Oct 30 18:47:07.098385 ip-10-0-139-33 kubenswrapper[5704]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:47:07.098385 ip-10-0-139-33 kubenswrapper[5704]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Oct 30 18:47:07.098385 ip-10-0-139-33 kubenswrapper[5704]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:47:07.098385 ip-10-0-139-33 kubenswrapper[5704]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Oct 30 18:47:07.100361 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100290    5704 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:47:07.100361 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100361    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100365    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100369    5704 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100373    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100376    5704 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100380    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100383    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100386    5704 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100388    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100391    5704 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100394    5704 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100396    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100404    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100408    5704 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100410    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100413    5704 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100416    5704 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100419    5704 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100421    5704 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:47:07.100440 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100424    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100426    5704 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100429    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100431    5704 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100434    5704 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100437    5704 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100439    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100442    5704 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100445    5704 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100449    5704 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100453    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100457    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100460    5704 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100463    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100467    5704 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100469    5704 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100472    5704 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100474    5704 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100477    5704 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:47:07.100928 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100480    5704 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100482    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100485    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100487    5704 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100490    5704 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100492    5704 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100495    5704 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100498    5704 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100501    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100503    5704 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100505    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100508    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100510    5704 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100513    5704 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100516    5704 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100519    5704 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100522    5704 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100524    5704 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100527    5704 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100529    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:47:07.101434 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100532    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100534    5704 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100537    5704 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100539    5704 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100541    5704 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100544    5704 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100546    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100549    5704 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100551    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100554    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100556    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100559    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100561    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100564    5704 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100566    5704 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100569    5704 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100571    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100574    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100576    5704 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100579    5704 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:47:07.101974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100582    5704 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100584    5704 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100587    5704 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100591    5704 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100595    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100598    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100605    5704 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100608    5704 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100611    5704 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100613    5704 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100616    5704 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100618    5704 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100621    5704 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100624    5704 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100626    5704 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100628    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100631    5704 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100634    5704 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100636    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:47:07.102515 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100639    5704 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100641    5704 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100644    5704 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100647    5704 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100650    5704 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100652    5704 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100655    5704 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.100657    5704 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101425    5704 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101433    5704 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101436    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101439    5704 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101442    5704 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101446    5704 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101450    5704 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101453    5704 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101456    5704 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101458    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101462    5704 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101468    5704 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:47:07.102994 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101471    5704 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101473    5704 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101476    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101479    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101482    5704 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101484    5704 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101487    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101490    5704 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101492    5704 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101495    5704 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101497    5704 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101500    5704 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101503    5704 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101505    5704 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101508    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101511    5704 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101514    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101516    5704 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101519    5704 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101521    5704 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:47:07.103511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101524    5704 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101526    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101529    5704 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101531    5704 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101534    5704 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101536    5704 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101539    5704 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101542    5704 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101544    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101547    5704 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101550    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101554    5704 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101557    5704 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101560    5704 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101562    5704 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101565    5704 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101568    5704 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101570    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101573    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:47:07.104026 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101575    5704 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101578    5704 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101581    5704 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101583    5704 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101586    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101590    5704 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101593    5704 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101597    5704 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101599    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101602    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101605    5704 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101607    5704 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101610    5704 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101612    5704 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101615    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101617    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101620    5704 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101622    5704 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101625    5704 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:47:07.104517 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101627    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101630    5704 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101632    5704 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101635    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101638    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101641    5704 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101644    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101647    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101649    5704 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101652    5704 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101654    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101657    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101659    5704 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101662    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101664    5704 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101667    5704 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101671    5704 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101675    5704 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101679    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:47:07.105013 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101683    5704 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101686    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101688    5704 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101691    5704 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101693    5704 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101696    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101698    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101700    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101703    5704 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101705    5704 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101708    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101710    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101713    5704 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101715    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101717    5704 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101720    5704 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.101723    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102219    5704 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102227    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102230    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:47:07.105503 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102232    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102235    5704 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102240    5704 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102243    5704 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102246    5704 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102249    5704 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102252    5704 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102254    5704 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102257    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102260    5704 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102262    5704 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102265    5704 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102268    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102270    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102273    5704 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102275    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102278    5704 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102281    5704 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102283    5704 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:47:07.106019 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102286    5704 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102289    5704 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102291    5704 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102294    5704 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102296    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102299    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102301    5704 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102304    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102306    5704 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102311    5704 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102314    5704 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102318    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102321    5704 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102323    5704 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102326    5704 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102328    5704 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102331    5704 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102334    5704 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102336    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102339    5704 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:47:07.106521 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102341    5704 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102344    5704 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102346    5704 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102349    5704 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102351    5704 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102354    5704 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102357    5704 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102359    5704 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102362    5704 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102364    5704 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102366    5704 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102369    5704 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102372    5704 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102374    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102377    5704 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102379    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102382    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102384    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102386    5704 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102389    5704 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:47:07.107178 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102392    5704 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102395    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102398    5704 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102403    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102406    5704 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102408    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102411    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102413    5704 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102416    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102418    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102421    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102424    5704 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102426    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102429    5704 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102431    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102434    5704 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102436    5704 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102439    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102441    5704 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:47:07.107743 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102444    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102447    5704 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102449    5704 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102452    5704 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102455    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102457    5704 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102460    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102463    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102465    5704 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102467    5704 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102470    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102472    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102475    5704 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102478    5704 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102481    5704 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102484    5704 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102489    5704 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102491    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102494    5704 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102497    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:47:07.108526 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102499    5704 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102502    5704 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102504    5704 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102507    5704 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.102511    5704 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102599    5704 flags.go:64] FLAG: --address="0.0.0.0"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102606    5704 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102613    5704 flags.go:64] FLAG: --anonymous-auth="true"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102620    5704 flags.go:64] FLAG: --application-metrics-count-limit="100"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102626    5704 flags.go:64] FLAG: --authentication-token-webhook="false"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102631    5704 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102636    5704 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102643    5704 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102646    5704 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102649    5704 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102653    5704 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102656    5704 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102660    5704 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102663    5704 flags.go:64] FLAG: --cgroup-root=""
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102665    5704 flags.go:64] FLAG: --cgroups-per-qos="true"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102669    5704 flags.go:64] FLAG: --client-ca-file=""
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102672    5704 flags.go:64] FLAG: --cloud-provider="external"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102675    5704 flags.go:64] FLAG: --cluster-dns="[]"
Oct 30 18:47:07.109411 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102678    5704 flags.go:64] FLAG: --cluster-domain=""
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102681    5704 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102684    5704 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102688    5704 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102691    5704 flags.go:64] FLAG: --container-log-max-files="5"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102697    5704 flags.go:64] FLAG: --container-log-max-size="10Mi"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102701    5704 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102704    5704 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102707    5704 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102710    5704 flags.go:64] FLAG: --contention-profiling="false"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102713    5704 flags.go:64] FLAG: --cpu-cfs-quota="true"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102715    5704 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102719    5704 flags.go:64] FLAG: --cpu-manager-policy="none"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102721    5704 flags.go:64] FLAG: --cpu-manager-policy-options=""
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102726    5704 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102729    5704 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102732    5704 flags.go:64] FLAG: --enable-debugging-handlers="true"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102735    5704 flags.go:64] FLAG: --enable-load-reader="false"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102738    5704 flags.go:64] FLAG: --enable-server="true"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102741    5704 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102745    5704 flags.go:64] FLAG: --event-burst="100"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102748    5704 flags.go:64] FLAG: --event-qps="50"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102751    5704 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102754    5704 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Oct 30 18:47:07.109993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102757    5704 flags.go:64] FLAG: --eviction-hard=""
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102761    5704 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102763    5704 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102766    5704 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102769    5704 flags.go:64] FLAG: --eviction-soft=""
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102772    5704 flags.go:64] FLAG: --eviction-soft-grace-period=""
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102775    5704 flags.go:64] FLAG: --exit-on-lock-contention="false"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102778    5704 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102781    5704 flags.go:64] FLAG: --experimental-mounter-path=""
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102784    5704 flags.go:64] FLAG: --fail-cgroupv1="true"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102787    5704 flags.go:64] FLAG: --fail-swap-on="true"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102790    5704 flags.go:64] FLAG: --feature-gates=""
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102795    5704 flags.go:64] FLAG: --file-check-frequency="20s"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102801    5704 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102804    5704 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102808    5704 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102811    5704 flags.go:64] FLAG: --healthz-port="10248"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102814    5704 flags.go:64] FLAG: --help="false"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102817    5704 flags.go:64] FLAG: --hostname-override="ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102820    5704 flags.go:64] FLAG: --housekeeping-interval="10s"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102823    5704 flags.go:64] FLAG: --http-check-frequency="20s"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102826    5704 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102829    5704 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Oct 30 18:47:07.110650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102833    5704 flags.go:64] FLAG: --image-gc-high-threshold="85"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102836    5704 flags.go:64] FLAG: --image-gc-low-threshold="80"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102838    5704 flags.go:64] FLAG: --image-service-endpoint=""
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102841    5704 flags.go:64] FLAG: --kernel-memcg-notification="false"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102844    5704 flags.go:64] FLAG: --kube-api-burst="100"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102847    5704 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102850    5704 flags.go:64] FLAG: --kube-api-qps="50"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102853    5704 flags.go:64] FLAG: --kube-reserved=""
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102856    5704 flags.go:64] FLAG: --kube-reserved-cgroup=""
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102859    5704 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102862    5704 flags.go:64] FLAG: --kubelet-cgroups=""
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102865    5704 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102868    5704 flags.go:64] FLAG: --lock-file=""
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102871    5704 flags.go:64] FLAG: --log-cadvisor-usage="false"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102874    5704 flags.go:64] FLAG: --log-flush-frequency="5s"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102877    5704 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102882    5704 flags.go:64] FLAG: --log-json-split-stream="false"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102885    5704 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102888    5704 flags.go:64] FLAG: --log-text-split-stream="false"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102891    5704 flags.go:64] FLAG: --logging-format="text"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102894    5704 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102898    5704 flags.go:64] FLAG: --make-iptables-util-chains="true"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102902    5704 flags.go:64] FLAG: --manifest-url=""
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102905    5704 flags.go:64] FLAG: --manifest-url-header=""
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102910    5704 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Oct 30 18:47:07.111260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102914    5704 flags.go:64] FLAG: --max-open-files="1000000"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102918    5704 flags.go:64] FLAG: --max-pods="110"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102920    5704 flags.go:64] FLAG: --maximum-dead-containers="-1"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102923    5704 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102926    5704 flags.go:64] FLAG: --memory-manager-policy="None"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102929    5704 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102933    5704 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102935    5704 flags.go:64] FLAG: --node-ip="0.0.0.0"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102938    5704 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102945    5704 flags.go:64] FLAG: --node-status-max-images="50"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102948    5704 flags.go:64] FLAG: --node-status-update-frequency="10s"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102951    5704 flags.go:64] FLAG: --oom-score-adj="-999"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102954    5704 flags.go:64] FLAG: --pod-cidr=""
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102957    5704 flags.go:64] FLAG: --pod-manifest-path=""
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102960    5704 flags.go:64] FLAG: --pod-max-pids="-1"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102963    5704 flags.go:64] FLAG: --pods-per-core="0"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102965    5704 flags.go:64] FLAG: --port="10250"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102969    5704 flags.go:64] FLAG: --protect-kernel-defaults="false"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102971    5704 flags.go:64] FLAG: --provider-id="aws:///us-west-2a/i-070fe8a2c1d4a5301"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102975    5704 flags.go:64] FLAG: --qos-reserved=""
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102977    5704 flags.go:64] FLAG: --read-only-port="10255"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102980    5704 flags.go:64] FLAG: --register-node="true"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102983    5704 flags.go:64] FLAG: --register-with-taints=""
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102987    5704 flags.go:64] FLAG: --registry-burst="10"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102990    5704 flags.go:64] FLAG: --registry-qps="5"
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102993    5704 flags.go:64] FLAG: --reserved-cpus=""
Oct 30 18:47:07.111987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.102996    5704 flags.go:64] FLAG: --reserved-memory=""
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103003    5704 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103007    5704 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103010    5704 flags.go:64] FLAG: --rotate-certificates="false"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103013    5704 flags.go:64] FLAG: --rotate-server-certificates="false"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103016    5704 flags.go:64] FLAG: --runonce="false"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103019    5704 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103023    5704 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103026    5704 flags.go:64] FLAG: --seccomp-default="false"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103040    5704 flags.go:64] FLAG: --serialize-image-pulls="true"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103043    5704 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103046    5704 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103049    5704 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103052    5704 flags.go:64] FLAG: --storage-driver-password="root"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103055    5704 flags.go:64] FLAG: --storage-driver-secure="false"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103058    5704 flags.go:64] FLAG: --storage-driver-table="stats"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103061    5704 flags.go:64] FLAG: --storage-driver-user="root"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103064    5704 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103067    5704 flags.go:64] FLAG: --sync-frequency="1m0s"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103070    5704 flags.go:64] FLAG: --system-cgroups=""
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103072    5704 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103078    5704 flags.go:64] FLAG: --system-reserved-cgroup=""
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103081    5704 flags.go:64] FLAG: --tls-cert-file=""
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103083    5704 flags.go:64] FLAG: --tls-cipher-suites="[]"
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103087    5704 flags.go:64] FLAG: --tls-min-version=""
Oct 30 18:47:07.112649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103089    5704 flags.go:64] FLAG: --tls-private-key-file=""
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103092    5704 flags.go:64] FLAG: --topology-manager-policy="none"
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103095    5704 flags.go:64] FLAG: --topology-manager-policy-options=""
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103098    5704 flags.go:64] FLAG: --topology-manager-scope="container"
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103101    5704 flags.go:64] FLAG: --v="2"
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103105    5704 flags.go:64] FLAG: --version="false"
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103109    5704 flags.go:64] FLAG: --vmodule=""
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103115    5704 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103118    5704 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103245    5704 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103248    5704 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103251    5704 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103254    5704 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103257    5704 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103260    5704 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103263    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103265    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103268    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103271    5704 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103274    5704 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103276    5704 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103279    5704 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:47:07.113575 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103281    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103284    5704 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103287    5704 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103289    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103292    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103294    5704 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103297    5704 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103300    5704 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103302    5704 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103304    5704 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103307    5704 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103310    5704 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103312    5704 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103315    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103318    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103320    5704 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103323    5704 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103327    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103330    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:47:07.114269 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103332    5704 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103335    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103337    5704 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103341    5704 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103344    5704 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103347    5704 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103350    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103353    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103355    5704 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103358    5704 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103361    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103363    5704 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103366    5704 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103368    5704 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103371    5704 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103373    5704 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103376    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103378    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103381    5704 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:47:07.114815 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103385    5704 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103389    5704 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103392    5704 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103395    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103397    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103400    5704 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103402    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103405    5704 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103407    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103410    5704 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103413    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103416    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103419    5704 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103421    5704 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103424    5704 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103427    5704 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103429    5704 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103432    5704 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103434    5704 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103437    5704 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:47:07.115383 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103440    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103442    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103445    5704 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103448    5704 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103450    5704 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103453    5704 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103455    5704 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103458    5704 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103460    5704 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103463    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103466    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103470    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103472    5704 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103475    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103478    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103480    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103483    5704 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103485    5704 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103488    5704 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:47:07.115927 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103491    5704 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103493    5704 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103496    5704 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103498    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103502    5704 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103504    5704 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103507    5704 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103510    5704 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103513    5704 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103515    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103518    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103521    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103524    5704 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103526    5704 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103529    5704 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103532    5704 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:47:07.116518 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.103558    5704 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103591    5704 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103594    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103597    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103600    5704 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103602    5704 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103605    5704 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103608    5704 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103611    5704 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103615    5704 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103617    5704 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103620    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103622    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103625    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103627    5704 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103630    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103633    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103635    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103638    5704 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:47:07.117022 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103640    5704 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103644    5704 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103647    5704 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103650    5704 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103652    5704 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103655    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103657    5704 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103660    5704 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103663    5704 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103665    5704 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103668    5704 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103670    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103673    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103676    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103678    5704 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103681    5704 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103684    5704 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103686    5704 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103689    5704 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103691    5704 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:47:07.117566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103694    5704 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103696    5704 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103700    5704 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103703    5704 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103706    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103708    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103711    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103713    5704 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103716    5704 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103718    5704 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103721    5704 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103724    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103727    5704 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103730    5704 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103733    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103735    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103738    5704 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103741    5704 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103743    5704 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103746    5704 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:47:07.118089 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103748    5704 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103751    5704 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103754    5704 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103756    5704 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103759    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103762    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103765    5704 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103767    5704 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103770    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103772    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103775    5704 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103777    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103780    5704 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103784    5704 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103791    5704 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103794    5704 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103797    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103800    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103802    5704 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:47:07.118615 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103805    5704 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103807    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103810    5704 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103812    5704 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103815    5704 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103818    5704 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103821    5704 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103824    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103827    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103829    5704 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103833    5704 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103837    5704 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103840    5704 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103843    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103846    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103848    5704 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103851    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103853    5704 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103856    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103858    5704 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:47:07.119102 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103861    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103864    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103866    5704 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103869    5704 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103872    5704 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103875    5704 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103878    5704 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103882    5704 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.103884    5704 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.108182    5704 server.go:525] "Kubelet version" kubeletVersion="v1.35.9"
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.108202    5704 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.108227    5704 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.108234    5704 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108392    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108399    5704 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108403    5704 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108408    5704 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108413    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108417    5704 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108427    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:47:07.119629 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108432    5704 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108436    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108440    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108444    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108448    5704 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108453    5704 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108457    5704 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108464    5704 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108469    5704 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108474    5704 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108479    5704 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108488    5704 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108493    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108498    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108502    5704 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108506    5704 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108511    5704 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108554    5704 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108585    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:47:07.120311 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108592    5704 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108597    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108601    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108606    5704 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108611    5704 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108615    5704 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108619    5704 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108624    5704 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108629    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108647    5704 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108663    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108684    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108814    5704 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108820    5704 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108823    5704 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108827    5704 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108830    5704 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108834    5704 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108836    5704 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108840    5704 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:47:07.120809 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108843    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108845    5704 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108848    5704 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108851    5704 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108854    5704 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108856    5704 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108859    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108862    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108864    5704 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108867    5704 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108877    5704 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108880    5704 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108883    5704 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108886    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108888    5704 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108891    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108894    5704 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108897    5704 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108900    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108903    5704 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:47:07.121475 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108906    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108908    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108911    5704 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108913    5704 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108916    5704 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108919    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108921    5704 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108925    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108927    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108930    5704 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108932    5704 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108936    5704 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108938    5704 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108941    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108944    5704 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108946    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108949    5704 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108952    5704 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108954    5704 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:47:07.121969 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108957    5704 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108960    5704 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108965    5704 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108969    5704 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108973    5704 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108976    5704 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108979    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108982    5704 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108984    5704 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108987    5704 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108990    5704 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108992    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108995    5704 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.108997    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109002    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109004    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109007    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109009    5704 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109012    5704 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109014    5704 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:47:07.122464 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109018    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.109076    5704 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109210    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109216    5704 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109219    5704 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109222    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109226    5704 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109231    5704 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109234    5704 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109237    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109240    5704 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109244    5704 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109247    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109250    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109253    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:47:07.122950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109257    5704 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109260    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109264    5704 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109267    5704 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109270    5704 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109273    5704 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109275    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109279    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109281    5704 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109284    5704 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109287    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109289    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109292    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109294    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109297    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109299    5704 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109302    5704 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109305    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109307    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:47:07.123335 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109310    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109313    5704 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109316    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109319    5704 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109321    5704 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109324    5704 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109328    5704 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109330    5704 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109333    5704 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109337    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109340    5704 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109342    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109345    5704 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109348    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109350    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109353    5704 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109356    5704 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109358    5704 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109361    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109364    5704 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:47:07.123801 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109384    5704 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109388    5704 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109390    5704 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109393    5704 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109396    5704 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109399    5704 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109401    5704 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109404    5704 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109406    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109409    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109412    5704 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109415    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109418    5704 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109421    5704 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109424    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109427    5704 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109429    5704 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109432    5704 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109434    5704 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109438    5704 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:47:07.124308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109440    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109443    5704 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109446    5704 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109448    5704 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109451    5704 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109454    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109456    5704 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109459    5704 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109462    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109464    5704 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109467    5704 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109469    5704 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109472    5704 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109474    5704 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109477    5704 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109480    5704 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109482    5704 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109485    5704 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109487    5704 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109490    5704 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:47:07.124799 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109493    5704 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109495    5704 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109497    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109501    5704 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109503    5704 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109506    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109509    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109511    5704 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109514    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109516    5704 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109519    5704 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109521    5704 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109524    5704 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109527    5704 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.109552    5704 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]}
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109582    5704 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles
Oct 30 18:47:07.125462 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109586    5704 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109589    5704 feature_gate.go:386] unrecognized feature gate: EVPN
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109592    5704 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109595    5704 feature_gate.go:386] unrecognized feature gate: Example
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109598    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109601    5704 feature_gate.go:386] unrecognized feature gate: ShortCertRotation
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109603    5704 feature_gate.go:386] unrecognized feature gate: EventTTL
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109605    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109609    5704 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109613    5704 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109616    5704 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109619    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109622    5704 feature_gate.go:386] unrecognized feature gate: TLSAdherence
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109626    5704 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109628    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109631    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109634    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109637    5704 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109641    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Oct 30 18:47:07.125868 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109643    5704 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109646    5704 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109649    5704 feature_gate.go:386] unrecognized feature gate: NewOLM
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109652    5704 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109654    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDC
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109657    5704 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109659    5704 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109662    5704 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109665    5704 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109667    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109670    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109673    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109676    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109678    5704 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109680    5704 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109683    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109685    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109688    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109691    5704 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement
Oct 30 18:47:07.126565 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109693    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109695    5704 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109698    5704 feature_gate.go:386] unrecognized feature gate: Example2
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109700    5704 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109703    5704 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109706    5704 feature_gate.go:386] unrecognized feature gate: DualReplica
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109708    5704 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109711    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109714    5704 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109716    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109719    5704 feature_gate.go:386] unrecognized feature gate: NoOverlayMode
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109721    5704 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109724    5704 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109727    5704 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109729    5704 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109732    5704 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109735    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109737    5704 feature_gate.go:386] unrecognized feature gate: SignatureStores
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109740    5704 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109743    5704 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator
Oct 30 18:47:07.127308 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109745    5704 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109748    5704 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109750    5704 feature_gate.go:386] unrecognized feature gate: OVNObservability
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109753    5704 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109756    5704 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109759    5704 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109761    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109766    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109769    5704 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109773    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109778    5704 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109792    5704 feature_gate.go:386] unrecognized feature gate: OSStreams
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109824    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109828    5704 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109830    5704 feature_gate.go:386] unrecognized feature gate: NetworkConnect
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109834    5704 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109838    5704 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109842    5704 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109846    5704 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109850    5704 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall
Oct 30 18:47:07.127836 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109854    5704 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109858    5704 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109862    5704 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109865    5704 feature_gate.go:386] unrecognized feature gate: KMSEncryption
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109868    5704 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109871    5704 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109877    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109880    5704 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109883    5704 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109885    5704 feature_gate.go:386] unrecognized feature gate: DNSNameResolver
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109888    5704 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109891    5704 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109893    5704 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109897    5704 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109900    5704 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109903    5704 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109905    5704 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109909    5704 feature_gate.go:386] unrecognized feature gate: InsightsConfig
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109912    5704 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109914    5704 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall
Oct 30 18:47:07.128332 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109918    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109922    5704 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109927    5704 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109931    5704 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109935    5704 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109940    5704 feature_gate.go:386] unrecognized feature gate: UpgradeStatus
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:47:07.109942    5704 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.110028    5704 server.go:957] "Client rotation is on, will bootstrap in background"
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.111136    5704 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary"
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.111192    5704 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem"
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.111358    5704 server.go:1014] "Starting client certificate rotation"
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.111507    5704 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.111552    5704 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:40:56 +0000 UTC" deadline="2028-04-24 16:08:18.382283647 +0000 UTC"
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.111579    5704 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13005h21m11.2707063s"
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.111824    5704 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.112927    5704 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Oct 30 18:47:07.128841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.114364    5704 log.go:25] "Validated CRI v1 runtime API"
Oct 30 18:47:07.129310 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.116156    5704 log.go:25] "Validated CRI v1 image API"
Oct 30 18:47:07.129310 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.116894    5704 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Oct 30 18:47:07.129310 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.118916    5704 fs.go:135] Filesystem UUIDs: map[74da84f2-0b61-4b9f-b60a-82c89b126057:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 cfbc3d41-e7f3-4785-8e52-0150bb65b235:/dev/nvme0n1p3]
Oct 30 18:47:07.129437 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.118935    5704 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0143f87b71fb5861a7461820eb8d145a3d7692cf0dc40802b3fbc3efe002ded1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0143f87b71fb5861a7461820eb8d145a3d7692cf0dc40802b3fbc3efe002ded1/userdata/shm major:0 minor:224 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/0e7ff7bd6e351621e118ef488ba19ec4e7a6c27eb2d14ae71d3350f49d7fbbc0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/0e7ff7bd6e351621e118ef488ba19ec4e7a6c27eb2d14ae71d3350f49d7fbbc0/userdata/shm major:0 minor:205 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/17bf4ea6d518f6a340d2f3cc1e666c8fd718ff015e8c61fb7cb9ac843d44273c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/17bf4ea6d518f6a340d2f3cc1e666c8fd718ff015e8c61fb7cb9ac843d44273c/userdata/shm major:0 minor:80 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1b28ccf8718b1b23df3e6129770c007aef7bbf94cb020f329fa3135fbe247c8d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1b28ccf8718b1b23df3e6129770c007aef7bbf94cb020f329fa3135fbe247c8d/userdata/shm major:0 minor:343 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2c86a6df7933fd5687cc84bb4c30258050ccf42bd55d0f50f7fcdc89fc7f428d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2c86a6df7933fd5687cc84bb4c30258050ccf42bd55d0f50f7fcdc89fc7f428d/userdata/shm major:0 minor:68 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/3725fc11d97fb1eeafa752dfadf0018bef6aeedbde6ca23e8d8b731fb7f0f2d4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/3725fc11d97fb1eeafa752dfadf0018bef6aeedbde6ca23e8d8b731fb7f0f2d4/userdata/shm major:0 minor:454 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4411f88293ac8985be9ff0bf0c171ca23852e5509dc762a19cce13c841ae8c4d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4411f88293ac8985be9ff0bf0c171ca23852e5509dc762a19cce13c841ae8c4d/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/69f9ba226e0a06bb9ced9993a88533d9b4a09b10eb66144081a213f1390cfed8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/69f9ba226e0a06bb9ced9993a88533d9b4a09b10eb66144081a213f1390cfed8/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6b05236719987d7322ada67217d827746076b2352df5d72bb8da7b6b5135b9fa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6b05236719987d7322ada67217d827746076b2352df5d72bb8da7b6b5135b9fa/userdata/shm major:0 minor:357 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6b9097d4a86d0d7d8cd8544da83af7d44980b474ef763e394400775fe3d1538e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6b9097d4a86d0d7d8cd8544da83af7d44980b474ef763e394400775fe3d1538e/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/7bc50e16469aee5af6b8dab32fa3c80a8f1923d1eb0f3763e875fc6c42bec2b8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/7bc50e16469aee5af6b8dab32fa3c80a8f1923d1eb0f3763e875fc6c42bec2b8/userdata/shm major:0 minor:352 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/80ff3b20e76a7e6edd5a96faa8e3e9373e5b4fd8d417efa5e26120fc44e52d74/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/80ff3b20e76a7e6edd5a96faa8e3e9373e5b4fd8d417efa5e26120fc44e52d74/userdata/shm major:0 minor:266 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/8dafede46b3521ea090abb5b0da5a438bed9a6677568536d854d1baed27fec81/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/8dafede46b3521ea090abb5b0da5a438bed9a6677568536d854d1baed27fec81/userdata/shm major:0 minor:274 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a5dcaf13f72dae8ace0899f1b37e8f9af530604a73d38e414d2bdf39ba069334/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a5dcaf13f72dae8ace0899f1b37e8f9af530604a73d38e414d2bdf39ba069334/userdata/shm major:0 minor:331 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/aad11b046a3a03af247ae3e4b2047008c22032eb02ce4ad94ea40e7b52a05268/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/aad11b046a3a03af247ae3e4b2047008c22032eb02ce4ad94ea40e7b52a05268/userdata/shm major:0 minor:209 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ab2f241cbd27f93527f68f203a29005c910414f14a4eeedc9e501ddbba116f98/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ab2f241cbd27f93527f68f203a29005c910414f14a4eeedc9e501ddbba116f98/userdata/shm major:0 minor:81 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d3e3fcdbe90125127b344c0c4a23486da5bef2f20e0c7df390a87b46b0c261d9/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d3e3fcdbe90125127b344c0c4a23486da5bef2f20e0c7df390a87b46b0c261d9/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/daa36dfdd6a1af71a65334aa018896fed1f9e008030b0ff3f29a8bcdf60f895b/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/daa36dfdd6a1af71a65334aa018896fed1f9e008030b0ff3f29a8bcdf60f895b/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/de7265384d17948ad084102cbb4ca2c97b44c67b5a062aba3e6c16d511b8d2a1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/de7265384d17948ad084102cbb4ca2c97b44c67b5a062aba3e6c16d511b8d2a1/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f2f6cee433b8f8ac45f4eaedafa95e02bc0cc4cfb7d070615acc3657d05f8d62/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f2f6cee433b8f8ac45f4eaedafa95e02bc0cc4cfb7d070615acc3657d05f8d62/userdata/shm major:0 minor:258 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f55989fc59f099067d253917e0f1844949c4e949832e54d51fe440249fa95219/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f55989fc59f099067d253917e0f1844949c4e949832e54d51fe440249fa95219/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f8e3edc927fdc1674f512ef088d22ea4aebb6823380e08537e85cf60933eec98/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f8e3edc927fdc1674f512ef088d22ea4aebb6823380e08537e85cf60933eec98/userdata/shm major:0 minor:452 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fc2294e8c03f9d3b1327c54fdfcc69e498a708099b0009121cf32543672797ae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fc2294e8c03f9d3b1327c54fdfcc69e498a708099b0009121cf32543672797ae/userdata/shm major:0 minor:366 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fe08928888a95c73bc82dca6980c49086a5b3bc8aab554f3ea99d38b5f5aefb4/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fe08928888a95c73bc82dca6980c49086a5b3bc8aab554f3ea99d38b5f5aefb4/userdata/shm major:0 minor:207 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0e786874-ce56-46f8-87f5-920dacacd9fc/volumes/kubernetes.io~projected/kube-api-access-mqxts:{mountpoint:/var/lib/kubelet/pods/0e786874-ce56-46f8-87f5-920dacacd9fc/volumes/kubernetes.io~projected/kube-api-access-mqxts major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0e786874-ce56-46f8-87f5-920dacacd9fc/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/0e786874-ce56-46f8-87f5-920dacacd9fc/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config major:0 minor:263 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0e786874-ce56-46f8-87f5-920dacacd9fc/volumes/kubernetes.io~secret/kube-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/0e786874-ce56-46f8-87f5-920dacacd9fc/volumes/kubernetes.io~secret/kube-state-metrics-tls major:0 minor:262 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11550a43-fc10-43e9-835c-ae7c8b21be03/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/11550a43-fc10-43e9-835c-ae7c8b21be03/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11550a43-fc10-43e9-835c-ae7c8b21be03/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/11550a43-fc10-43e9-835c-ae7c8b21be03/volumes/kubernetes.io~empty-dir/tmp major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/11550a43-fc10-43e9-835c-ae7c8b21be03/volumes/kubernetes.io~projected/kube-api-access-zdzp8:{mountpoint:/var/lib/kubelet/pods/11550a43-fc10-43e9-835c-ae7c8b21be03/volumes/kubernetes.io~projected/kube-api-access-zdzp8 major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f359854-4031-4cf7-9ef4-877e4a90d249/volumes/kubernetes.io~projected/kube-api-access-vsptj:{mountpoint:/var/lib/kubelet/pods/1f359854-4031-4cf7-9ef4-877e4a90d249/volumes/kubernetes.io~projected/kube-api-access-vsptj major:0 minor:351 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f359854-4031-4cf7-9ef4-877e4a90d249/volumes/kubernetes.io~secret/console-oauth-config:{mountpoint:/var/lib/kubelet/pods/1f359854-4031-4cf7-9ef4-877e4a90d249/volumes/kubernetes.io~secret/console-oauth-config major:0 minor:349 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/1f359854-4031-4cf7-9ef4-877e4a90d249/volumes/kubernetes.io~secret/console-serving-cert:{mountpoint:/var/lib/kubelet/pods/1f359854-4031-4cf7-9ef4-877e4a90d249/volumes/kubernetes.io~secret/console-serving-cert major:0 minor:350 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/263a55c9-0e5a-47bc-bec1-20cd01e12942/volumes/kubernetes.io~projected/kube-api-access-t9cjq:{mountpoint:/var/lib/kubelet/pods/263a55c9-0e5a-47bc-bec1-20cd01e12942/volumes/kubernetes.io~projected/kube-api-access-t9cjq major:0 minor:363 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/263a55c9-0e5a-47bc-bec1-20cd01e12942/volumes/kubernetes.io~secret/client-ca-bundle:{mountpoint:/var/lib/kubelet/pods/263a55c9-0e5a-47bc-bec1-20cd01e12942/volumes/kubernetes.io~secret/client-ca-bundle major:0 minor:360 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/263a55c9-0e5a-47bc-bec1-20cd01e12942/volumes/kubernetes.io~secret/secret-metrics-server-client-certs:{mountpoint:/var/lib/kubelet/pods/263a55c9-0e5a-47bc-bec1-20cd01e12942/volumes/kubernetes.io~secret/secret-metrics-server-client-certs major:0 minor:361 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/263a55c9-0e5a-47bc-bec1-20cd01e12942/volumes/kubernetes.io~secret/secret-metrics-server-tls:{mountpoint:/var/lib/kubelet/pods/263a55c9-0e5a-47bc-bec1-20cd01e12942/volumes/kubernetes.io~secret/secret-metrics-server-tls major:0 minor:362 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volume-subpaths/cluster-tls-config/alertmanager/10:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volume-subpaths/cluster-tls-config/alertmanager/10 major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volume-subpaths/web-config/alertmanager/9:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volume-subpaths/web-config/alertmanager/9 major:0 minor:304 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volume-subpaths/web-config/config-reloader/6:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volume-subpaths/web-config/config-reloader/6 major:0 minor:304 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volume-subpaths/web-config/init-config-reloader/6:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volume-subpaths/web-config/init-config-reloader/6 major:0 minor:304 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~empty-dir/config-out:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~empty-dir/config-out major:0 minor:305 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~projected/kube-api-access-8brjr:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~projected/kube-api-access-8brjr major:0 minor:309 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~projected/tls-assets:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~projected/tls-assets major:0 minor:306 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/cluster-tls-config:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/cluster-tls-config major:0 minor:295 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/config-volume:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/config-volume major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric major:0 minor:301 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web major:0 minor:300 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/secret-alertmanager-main-tls:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/secret-alertmanager-main-tls major:0 minor:302 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/web-config:{mountpoint:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/web-config major:0 minor:304 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3acfed5c-f071-4450-9e72-d923cc492d3f/volumes/kubernetes.io~projected/kube-api-access-btbrv:{mountpoint:/var/lib/kubelet/pods/3acfed5c-f071-4450-9e72-d923cc492d3f/volumes/kubernetes.io~projected/kube-api-access-btbrv major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3acfed5c-f071-4450-9e72-d923cc492d3f/volumes/kubernetes.io~secret/monitoring-plugin-cert:{mountpoint:/var/lib/kubelet/pods/3acfed5c-f071-4450-9e72-d923cc492d3f/volumes/kubernetes.io~secret/monitoring-plugin-cert major:0 minor:358 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a/volumes/kubernetes.io~projected/kube-api-access-pvkcs:{mountpoint:/var/lib/kubelet/pods/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a/volumes/kubernetes.io~projected/kube-api-access-pvkcs major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3d07d694-4e2c-401a-aeab-0c9c4f530e87/volumes/kubernetes.io~projected/kube-api-access-bp6fz:{mountpoint:/var/lib/kubelet/pods/3d07d694-4e2c-401a-aeab-0c9c4f530e87/volumes/kubernetes.io~projected/kube-api-access-bp6fz major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3d07d694-4e2c-401a-aeab-0c9c4f530e87/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/3d07d694-4e2c-401a-aeab-0c9c4f530e87/volumes/kubernetes.io~secret/metrics-tls major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42b2d2d6-78f0-457a-b182-09dd65dc3372/volumes/kubernetes.io~projected/kube-api-access-jtds7:{mountpoint:/var/lib/kubelet/pods/42b2d2d6-78f0-457a-b182-09dd65dc3372/volumes/kubernetes.io~projected/kube-api-access-jtds7 major:0 minor:257 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42b2d2d6-78f0-457a-b182-09dd65dc3372/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/42b2d2d6-78f0-457a-b182-09dd65dc3372/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config major:0 minor:256 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/42b2d2d6-78f0-457a-b182-09dd65dc3372/volumes/kubernetes.io~secret/openshift-state-metrics-tls:{mountpoint:/var/lib/kubelet/pods/42b2d2d6-78f0-457a-b182-09dd65dc3372/volumes/kubernetes.io~secret/openshift-state-metrics-tls major:0 minor:251 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4cb022e6-43bd-4d71-9935-f0e91130735b/volumes/kubernetes.io~projected/kube-api-access-vrrbp:{mountpoint:/var/lib/kubelet/pods/4cb022e6-43bd-4d71-9935-f0e91130735b/volumes/kubernetes.io~projected/kube-api-access-vrrbp major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4cb022e6-43bd-4d71-9935-f0e91130735b/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/4cb022e6-43bd-4d71-9935-f0e91130735b/volumes/kubernetes.io~secret/metrics-certs major:0 minor:451 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/652a5a58-ad99-49bb-b371-743c2816482d/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/652a5a58-ad99-49bb-b371-743c2816482d/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/652a5a58-ad99-49bb-b371-743c2816482d/volumes/kubernetes.io~projected/kube-api-access-r2g7p:{mountpoint:/var/lib/kubelet/pods/652a5a58-ad99-49bb-b371-743c2816482d/volumes/kubernetes.io~projected/kube-api-access-r2g7p major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/652a5a58-ad99-49bb-b371-743c2816482d/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/652a5a58-ad99-49bb-b371-743c2816482d/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/738e26c2-0aac-4df0-961d-8c7404c8d6ab/volumes/kubernetes.io~projected/kube-api-access-w6fjw:{mountpoint:/var/lib/kubelet/pods/738e26c2-0aac-4df0-961d-8c7404c8d6ab/volumes/kubernetes.io~projected/kube-api-access-w6fjw major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6/volumes/kubernetes.io~projected/kube-api-access-z2gh2:{mountpoint:/var/lib/kubelet/pods/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6/volumes/kubernetes.io~projected/kube-api-access-z2gh2 major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6/volumes/kubernetes.io~secret/cert major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9822bfc7-212f-4cf6-8d56-eb44c58c359b/volumes/kubernetes.io~projected/kube-api-access-nck5c:{mountpoint:/var/lib/kubelet/pods/9822bfc7-212f-4cf6-8d56-eb44c58c359b/volumes/kubernetes.io~projected/kube-api-access-nck5c major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~projected/kube-api-access-z2t26:{mountpoint:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~projected/kube-api-access-z2t26 major:0 minor:333 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-grpc-tls:{mountpoint:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-grpc-tls major:0 minor:341 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy:{mountpoint:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy major:0 minor:339 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics:{mountpoint:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics major:0 minor:342 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules:{mountpoint:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web:{mountpoint:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web major:0 minor:338 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-tls:{mountpoint:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-tls major:0 minor:340 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a6c94923-1962-425d-8e53-ad523e78619b/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/a6c94923-1962-425d-8e53-ad523e78619b/volumes/kubernetes.io~secret/agent-certs major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/b67fee5e-0972-4cfc-987d-b0ae79e6589d/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/b67fee5e-0972-4cfc-987d-b0ae79e6589d/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:447 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ca586ba2-2d05-45f8-aab9-127b93ea1c60/volumes/kubernetes.io~projected/kube-api-access-wfhmw:{mountpoint:/var/lib/kubelet/pods/ca586ba2-2d05-45f8-aab9-127b93ea1c60/volumes/kubernetes.io~projected/kube-api-access-wfhmw major:0 minor:264 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ca586ba2-2d05-45f8-aab9-127b93ea1c60/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/ca586ba2-2d05-45f8-aab9-127b93ea1c60/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config major:0 minor:260 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ca586ba2-2d05-45f8-aab9-127b93ea1c60/volumes/kubernetes.io~secret/node-exporter-tls:{mountpoint:/var/lib/kubelet/pods/ca586ba2-2d05-45f8-aab9-127b93ea1c60/volumes/kubernetes.io~secret/node-exporter-tls major:0 minor:261 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cc73f4b8-ee7e-4915-a883-e26fa87a89db/volumes/kubernetes.io~projected/kube-api-access-hg9z4:{mountpoint:/var/lib/kubelet/pods/cc73f4b8-ee7e-4915-a883-e26fa87a89db/volumes/kubernetes.io~projected/kube-api-access-hg9z4 major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cda0bc7c-777f-4965-b097-6b5f3b6bfc52/volumes/kubernetes.io~projected/kube-api-access-77jh2:{mountpoint:/var/lib/kubelet/pods/cda0bc7c-777f-4965-b097-6b5f3b6bfc52/volumes/kubernetes.io~projected/kube-api-access-77jh2 major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0/volumes/kubernetes.io~projected/kube-api-access-5229t:{mountpoint:/var/lib/kubelet/pods/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0/volumes/kubernetes.io~projected/kube-api-access-5229t major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ea684836-96a5-4d68-a414-c3db5de7564e/volumes/kubernetes.io~projected/kube-api-access-ghc57:{mountpoint:/var/lib/kubelet/pods/ea684836-96a5-4d68-a414-c3db5de7564e/volumes/kubernetes.io~projected/kube-api-access-ghc57 major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ecbb798a-2024-4984-89c1-56ad55912e77/volumes/kubernetes.io~projected/kube-api-access-fsb2s:{mountpoint:/var/lib/kubelet/pods/ecbb798a-2024-4984-89c1-56ad55912e77/volumes/kubernetes.io~projected/kube-api-access-fsb2s major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ecbb798a-2024-4984-89c1-56ad55912e77/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/ecbb798a-2024-4984-89c1-56ad55912e77/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:197 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/d364c6b27565fc5a158205224c9e10d5f45e2ae55ffadfc77c7dbe6e487b1c9a/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/0897b21ca7989f865379d73b837ed7daa894b4da5f7abe1b3b5a593412671a64/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/2c6b1cca86f2fba516d05402a22bcd1d8346de27cb1b109699b6e24791248648/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/7f48829471c400bda8dc7b20cf1bc94b136aff8e7ab7ef12e5f058832bd858d3/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/0d413acb9f77a6ec3eb81f899f6de51341ff186a61ea42e72236f1f269f5f3bb/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/ce0f512b73224d034a5c95e3e83393bf2c05dd3981d9d072b13207dc132ff4aa/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/9cdd3561be3b779038aa05526b33cce95c529e28eef4ff05310c40b9ab881802/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/a5b2e0cc9962a312545e6dcd4c910384f8f1c1e224546b4ded4491416a716e27/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/de56a0b7b6db6eac5f2dfe284b3d914cd251b31c0dd7c9006224f114a9e251b9/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/07273ed29e81b3d222d039691b4981fa350149e38966e92cdfdde86cf94b2045/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-136:{mountpoint:/var/lib/containers/storage/overlay/c0bd7f5a1321371f6d0fb12e940808643e2a915c7f4a862ff8c27d4858f974df/merged major:0 minor:136 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/faa782e76d45d277695102dc232523f222a80ba1802cb5b8c2cb507f04c04f79/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/2d44dc370890ec029d61556cb482bd5cf7e86d0a831ffb01a21dea4eb85c244b/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/60edbb145081262a8fde39e185c8bb31569b50b7bab731af2c5b6b007a6dfa3b/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-162:{mountpoint:/var/lib/containers/storage/overlay/bdfe4585cf1924797dac9bb5541ce4c9db707cd6b67c8029c355e060ebc89a6f/merged major:0 minor:162 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/4c7d1e6d1b4f57d35b3ab68145cab589ded8fc99fe9b48871bac3f2bd0816dc1/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/af1ced6ef233ae83a449f5eb63217651d04e8db7583c0b8dc4c51292403e5702/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/cee8aaac1f46e406a3fc67eabd7af987744ded7f420626c4a7f8d5ca68b0560a/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/0e4e99db9261a13147c1c313535fddfcad88911509a47b52ef5022f1f47a1045/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/e853cefaa1b8053893bb579eb33c642456c65e68352e4f253bd61cd3ac2d4102/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/95fece39448289e4336667665ec83f03d7a4d6d702eeadf784a1305cf9cc1c22/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-211:{mountpoint:/var/lib/containers/storage/overlay/2eed7532ce34d334a98494d4237363f4b3ed206dd9932f56754f66afad22cf17/merged major:0 minor:211 fsType:overlay blockSize:0} overlay_0-213:{mountpoint:/var/lib/containers/storage/overlay/b6e326b8f7a7f2ca7dba96cd44c7b0ce47e38559216efe0569270e61c306f827/merged major:0 minor:213 fsType:overlay blockSize:0} overlay_0-215:{mountpoint:/var/lib/containers/storage/overlay/a99d857f544f350297947ae1838bb207e0cd0ce0c21e9fdd33b62440f2884b1d/merged major:0 minor:215 fsType:overlay blockSize:0} overlay_0-217:{mountpoint:/var/lib/containers/storage/overlay/2de1ea1049f7211d86b8be3d82c5fb8546d5526ff88aa808af222c4dcd5896a2/merged major:0 minor:217 fsType:overlay blockSize:0} overlay_0-226:{mountpoint:/var/lib/containers/storage/overlay/0e7bf2036218be384d4703f554ef35c693e434b651d4f8706326bfb439f06475/merged major:0 minor:226 fsType:overlay blockSize:0} overlay_0-228:{mountpoint:/var/lib/containers/storage/overlay/91d3715f628c1ba6c11a1421afb63e74f571e97726c26a2d6514e32af6fe7bf0/merged major:0 minor:228 fsType:overlay blockSize:0} overlay_0-230:{mountpoint:/var/lib/containers/storage/overlay/9ba2a5a776f7e0a7b287e8246d89904072db77db0c1e6f6e3dbabdad5b8c77c8/merged major:0 minor:230 fsType:overlay blockSize:0} overlay_0-235:{mountpoint:/var/lib/containers/storage/overlay/6a366fb88b07da078f71aeea321223a792364c4664f711b7b9a0c985a682ea09/merged major:0 minor:235 fsType:overlay blockSize:0} overlay_0-241:{mountpoint:/var/lib/containers/storage/overlay/1c94612fb246b4336e592966594fff81653daf5089737d6bbfa84369ec986a49/merged major:0 minor:241 fsType:overlay blockSize:0} overlay_0-246:{mountpoint:/var/lib/containers/storage/overlay/f2feaf7f7c138aecf9321c65bd38ede3b274dfa88305ffc0db50edbc475454fa/merged major:0 minor:246 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/c459b1ae003ae565f5c549be29586a677188bee26d4dfe0958375f29236e8b1a/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/5ae39a7c2bf27e4c79ee53947d0deb858cce555faa94419c8bacbe5ef4a9984b/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-272:{mountpoint:/var/lib/containers/storage/overlay/dcca13311e2e534ef4421c9f233912b3cfc7099270aaa950fd31a72c5aabf412/merged major:0 minor:272 fsType:overlay blockSize:0} overlay_0-276:{mountpoint:/var/lib/containers/storage/overlay/0ede5d6d75f622d7e12a7e2bd9aad1e341d7fd4b87d9a5655359c797face6a40/merged major:0 minor:276 fsType:overlay blockSize:0} overlay_0-282:{mountpoint:/var/lib/containers/storage/overlay/5d53836683194f3cc5514cba6ec4645224a5fb44528f915be41a06fb903badd2/merged major:0 minor:282 fsType:overlay blockSize:0} overlay_0-287:{mountpoint:/var/lib/containers/storage/overlay/c9260114c7cc646e3b821733d92b5dff385b4784c309cb86fe91c551cc04448b/merged major:0 minor:287 fsType:overlay blockSize:0} overlay_0-293:{mountpoint:/var/lib/containers/storage/overlay/cff0969c809c688f86c6a6f908aab0959647bc5755aaa948e3e58a6c228b25e3/merged major:0 minor:293 fsType:overlay blockSize:0} overlay_0-307:{mountpoint:/var/lib/containers/storage/overlay/b8f3170b20ec3f06ad7d023d07f694536d1ce7c2739236eea19e86068e603811/merged major:0 minor:307 fsType:overlay blockSize:0} overlay_0-310:{mountpoint:/var/lib/containers/storage/overlay/918d0473bb5bd39381428d41a5a9a7412c953705ba1ab39d3994bc90ebaf5ed6/merged major:0 minor:310 fsType:overlay blockSize:0} overlay_0-315:{mountpoint:/var/lib/containers/storage/overlay/f0c5d8a6f9b6ac15576e93ab66984115b87f0b1f8309a734e3571494fe4cb361/merged major:0 minor:315 fsType:overlay blockSize:0} overlay_0-320:{mountpoint:/var/lib/containers/storage/overlay/2fece714eb6c7098c7f037fad6e66f7b77055d3c38a15b559a099fd490c11ab2/merged major:0 minor:320 fsType:overlay blockSize:0} overlay_0-326:{mountpoint:/var/lib/containers/storage/overlay/a295564bd901531ad88ddf442e5b6dfbd2e729739c47e920a9bea6525268bd14/merged major:0 minor:326 fsType:overlay blockSize:0} overlay_0-345:{mountpoint:/var/lib/containers/storage/overlay/e0307f65e09472c3ed70287383e22306de5406fc395b0e60f9c41a4f038745ae/merged major:0 minor:345 fsType:overlay blockSize:0} overlay_0-347:{mountpoint:/var/lib/containers/storage/overlay/62a8a3e523b2c028b70bbdc42fc7e788c98196675725fbfe7be2aac3792880f5/merged major:0 minor:347 fsType:overlay blockSize:0} overlay_0-354:{mountpoint:/var/lib/containers/storage/overlay/0abf9b8ecd17b50a6e1afe3ae7cd20f852c128bb4f1468bdb58ea5d7022bdadb/merged major:0 minor:354 fsType:overlay blockSize:0} overlay_0-364:{mountpoint:/var/lib/containers/storage/overlay/caf48487b922daf50496e6737ca51d2249013ce529aaad032d869ff522c9540e/merged major:0 minor:364 fsType:overlay blockSize:0} overlay_0-368:{mountpoint:/var/lib/containers/storage/overlay/32e380fbcb0ffbce43872f0497bf276bab4e2eb448f1369aa8cd1a9a5e5bcdd1/merged major:0 minor:368 fsType:overlay blockSize:0} overlay_0-370:{mountpoint:/var/lib/containers/storage/overlay/55b8ce9b232c280e2702557da0573cdab23730ce21bb788146446a0a63ec46fd/merged major:0 minor:370 fsType:overlay blockSize:0} overlay_0-371:{mountpoint:/var/lib/containers/storage/overlay/46c88904d2365136706512eb52ab5c064dac6b5e30630cacc39ff10b540f411b/merged major:0 minor:371 fsType:overlay blockSize:0} overlay_0-378:{mountpoint:/var/lib/containers/storage/overlay/8446fcc0ea80f01d26471bab070d74aab25a76a603b8b8587f4d2a0ffb2a8031/merged major:0 minor:378 fsType:overlay blockSize:0} overlay_0-383:{mountpoint:/var/lib/containers/storage/overlay/d3788e7d7e1888a6319467bfd3b4fc650f4441ee78dccbb7226ec37552b8b3bd/merged major:0 minor:383 fsType:overlay blockSize:0} overlay_0-388:{mountpoint:/var/lib/containers/storage/overlay/5ac4c99a79a0731604b045eb00b5accfb2ba42e66a3a00f03fab243e83a2ec23/merged major:0 minor:388 fsType:overlay blockSize:0} overlay_0-394:{mountpoint:/var/lib/containers/storage/overlay/a0bd0b2631f4bc27e5b3303193b34395ae8413e42a0d830843f6ba89e47509c0/merged major:0 minor:394 fsType:overlay blockSize:0} overlay_0-399:{mountpoint:/var/lib/containers/storage/overlay/bdbf43ab5d6c940cbc4d09c2e1bda02c7ca2004521848d00a47fc6fc72a852ef/merged major:0 minor:399 fsType:overlay blockSize:0} overlay_0-404:{mountpoint:/var/lib/containers/storage/overlay/0ecb39529b5f123944628f5ec5fc56f953390af90c359c80088b01b18f011b56/merged major:0 minor:404 fsType:overlay blockSize:0} overlay_0-409:{mountpoint:/var/lib/containers/storage/overlay/e4886c614992163a03d1ab6ea957949acce0dbb50597af1dd4509e5feb726aa7/merged major:0 minor:409 fsType:overlay blockSize:0} overlay_0-415:{mountpoint:/var/lib/containers/storage/overlay/47d8e3d5ff01552bb969e305443613cc0991d1480821f573b5ae79c9c48cd7ec/merged major:0 minor:415 fsType:overlay blockSize:0} overlay_0-420:{mountpoint:/var/lib/containers/storage/overlay/eaead6cc208e2c79dcfc524fd9af59686004a86717ee0fc0673ba3b2a852018c/merged major:0 minor:420 fsType:overlay blockSize:0} overlay_0-422:{mountpoint:/var/lib/containers/storage/overlay/f304e687ef807eeaa556ad4d383fae70d80d4feb736446e1f6a0772ba5a295f2/merged major:0 minor:422 fsType:overlay blockSize:0} overlay_0-424:{mountpoint:/var/lib/containers/storage/overlay/c3cd1276a3d00364eb3aded52df870b2719c3ab4a9e0bf77681d08a08f667480/merged major:0 minor:424 fsType:overlay blockSize:0} overlay_0-425:{mountpoint:/var/lib/containers/storage/overlay/b66e175048cc9e87334dfbf949482d416876d0042bbed282645ea85833126b5c/merged major:0 minor:425 fsType:overlay blockSize:0} overlay_0-442:{mountpoint:/var/lib/containers/storage/overlay/dd5549e4d64d380dddbdcd81c5211ff579897d46d6a0bdbd8b328a8f765b3785/merged major:0 minor:442 fsType:overlay blockSize:0} overlay_0-456:{mountpoint:/var/lib/containers/storage/overlay/a668938cb418d9efe24bfec69974dfb420918fb02b9b2aef128759e14383ec31/merged major:0 minor:456 fsType:overlay blockSize:0} overlay_0-458:{mountpoint:/var/lib/containers/storage/overlay/2d52e67685635a4297288cc96975d1262319392315bf6c017a9593da7b9a3bb2/merged major:0 minor:458 fsType:overlay blockSize:0} overlay_0-460:{mountpoint:/var/lib/containers/storage/overlay/d9c8027d2525a27bf5c5a6f25857ec33dd7355050e5df7a636da02880800f88f/merged major:0 minor:460 fsType:overlay blockSize:0} overlay_0-462:{mountpoint:/var/lib/containers/storage/overlay/62b30ffef97b55a78faf397481a778ad6bb17ce3b9b478bca12706388ed5befb/merged major:0 minor:462 fsType:overlay blockSize:0} overlay_0-468:{mountpoint:/var/lib/containers/storage/overlay/faca9f6b1c1246aaa803cf2d3ae96601f62d443c621b722164b95edcc23769fa/merged major:0 minor:468 fsType:overlay blockSize:0} overlay_0-64:{mountpoint:/var/lib/containers/storage/overlay/4bc654fc1a8c3dd9b764e4d636d9df9253b7e7f72aa26c35aac3f346bfb928f8/merged major:0 minor:64 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/d606bbc3171b0d5f72e325b7a45979025056f77058ee07a1f932488f6df1528f/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/18fef901b0b5dc66e6e39bc6db3de4c82f1125c989c08cf8d40e898ff678f1d3/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/e75c90ddde1abb0f8f64623d795a8f9915dfb40c1e77e5c276cdaced1310d8cc/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-78:{mountpoint:/var/lib/containers/storage/overlay/7846cfea3332f8927c70cba32f2b42239973b0307acd7ffd764d5097ebd20738/merged major:0 minor:78 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/3cecbca857f833e3fbfd46783a12a1bc09125ef94fa65a0a6841088d8cd52720/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/12bc991bf667a05c8bb29e7e17d13a17c403da985fae5afc36d1301f9df71452/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/df4ff0d0d8d765b52a6b706065cc06b72e4e8bbc378f30a7621b8c5ac926e702/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/3751b9e151e02ec1f9e4ee9bbe9c155ce5012bbbc68ec05017e1a7a12833b4d2/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/d1e57313b6be0229744d0b64875b66466351d192551606d4142c9329ac354f26/merged major:0 minor:98 fsType:overlay blockSize:0}]
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.127928    5704 manager.go:217] Machine: {Timestamp:2026-10-30 18:47:07.127469753 +0000 UTC m=+0.073637647 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099135 MemoryCapacity:33164337152 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26c4b2fb4db98a9fb9b885e5d11722 SystemUUID:ec26c4b2-fb4d-b98a-9fb9-b885e5d11722 BootID:1fc1392e-c7ec-4db6-94a9-393ed7a98721 Filesystems:[{Device:overlay_0-64 DeviceMajor:0 DeviceMinor:64 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6b9097d4a86d0d7d8cd8544da83af7d44980b474ef763e394400775fe3d1538e/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-276 DeviceMajor:0 DeviceMinor:276 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6b05236719987d7322ada67217d827746076b2352df5d72bb8da7b6b5135b9fa/userdata/shm DeviceMajor:0 DeviceMinor:357 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volume-subpaths/web-config/config-reloader/6 DeviceMajor:0 DeviceMinor:304 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a5dcaf13f72dae8ace0899f1b37e8f9af530604a73d38e414d2bdf39ba069334/userdata/shm DeviceMajor:0 DeviceMinor:331 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a/volumes/kubernetes.io~projected/kube-api-access-pvkcs DeviceMajor:0 DeviceMinor:55 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6/volumes/kubernetes.io~projected/kube-api-access-z2gh2 DeviceMajor:0 DeviceMinor:202 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3d07d694-4e2c-401a-aeab-0c9c4f530e87/volumes/kubernetes.io~projected/kube-api-access-bp6fz DeviceMajor:0 DeviceMinor:203 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-293 DeviceMajor:0 DeviceMinor:293 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-425 DeviceMajor:0 DeviceMinor:425 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4cb022e6-43bd-4d71-9935-f0e91130735b/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:451 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-462 DeviceMajor:0 DeviceMinor:462 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/69f9ba226e0a06bb9ced9993a88533d9b4a09b10eb66144081a213f1390cfed8/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/7bc50e16469aee5af6b8dab32fa3c80a8f1923d1eb0f3763e875fc6c42bec2b8/userdata/shm DeviceMajor:0 DeviceMinor:352 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/263a55c9-0e5a-47bc-bec1-20cd01e12942/volumes/kubernetes.io~secret/client-ca-bundle DeviceMajor:0 DeviceMinor:360 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/4cb022e6-43bd-4d71-9935-f0e91130735b/volumes/kubernetes.io~projected/kube-api-access-vrrbp DeviceMajor:0 DeviceMinor:57 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2c86a6df7933fd5687cc84bb4c30258050ccf42bd55d0f50f7fcdc89fc7f428d/userdata/shm DeviceMajor:0 DeviceMinor:68 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3acfed5c-f071-4450-9e72-d923cc492d3f/volumes/kubernetes.io~secret/monitoring-plugin-cert DeviceMajor:0 DeviceMinor:358 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-442 DeviceMajor:0 DeviceMinor:442 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-320 DeviceMajor:0 DeviceMinor:320 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-394 DeviceMajor:0 DeviceMinor:394 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-409 DeviceMajor:0 DeviceMinor:409 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/738e26c2-0aac-4df0-961d-8c7404c8d6ab/volumes/kubernetes.io~projected/kube-api-access-w6fjw DeviceMajor:0 DeviceMinor:52 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-211 DeviceMajor:0 DeviceMinor:211 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-282 DeviceMajor:0 DeviceMinor:282 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/config-volume DeviceMajor:0 DeviceMinor:303 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/daa36dfdd6a1af71a65334aa018896fed1f9e008030b0ff3f29a8bcdf60f895b/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/8dafede46b3521ea090abb5b0da5a438bed9a6677568536d854d1baed27fec81/userdata/shm DeviceMajor:0 DeviceMinor:274 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-162 DeviceMajor:0 DeviceMinor:162 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d3e3fcdbe90125127b344c0c4a23486da5bef2f20e0c7df390a87b46b0c261d9/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fe08928888a95c73bc82dca6980c49086a5b3bc8aab554f3ea99d38b5f5aefb4/userdata/shm DeviceMajor:0 DeviceMinor:207 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ca586ba2-2d05-45f8-aab9-127b93ea1c60/volumes/kubernetes.io~secret/node-exporter-tls DeviceMajor:0 DeviceMinor:261 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ca586ba2-2d05-45f8-aab9-127b93ea1c60/volumes/kubernetes.io~projected/kube-api-access-wfhmw DeviceMajor:0 DeviceMinor:264 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:300 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-web DeviceMajor:0 DeviceMinor:338 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582168576 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/11550a43-fc10-43e9-835c-ae7c8b21be03/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:49 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/652a5a58-ad99-49bb-b371-743c2816482d/volumes/kubernetes.io~projected/kube-api-access-r2g7p DeviceMajor:0 DeviceMinor:59 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0e786874-ce56-46f8-87f5-920dacacd9fc/volumes/kubernetes.io~projected/kube-api-access-mqxts DeviceMajor:0 DeviceMinor:265 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy DeviceMajor:0 DeviceMinor:299 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volume-subpaths/web-config/init-config-reloader/6 DeviceMajor:0 DeviceMinor:304 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3d07d694-4e2c-401a-aeab-0c9c4f530e87/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:45 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-215 DeviceMajor:0 DeviceMinor:215 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0/volumes/kubernetes.io~projected/kube-api-access-5229t DeviceMajor:0 DeviceMinor:219 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-422 DeviceMajor:0 DeviceMinor:422 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/secret-alertmanager-main-tls DeviceMajor:0 DeviceMinor:302 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/263a55c9-0e5a-47bc-bec1-20cd01e12942/volumes/kubernetes.io~projected/kube-api-access-t9cjq DeviceMajor:0 DeviceMinor:363 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fc2294e8c03f9d3b1327c54fdfcc69e498a708099b0009121cf32543672797ae/userdata/shm DeviceMajor:0 DeviceMinor:366 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volume-subpaths/web-config/alertmanager/9 DeviceMajor:0 DeviceMinor:304 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-424 DeviceMajor:0 DeviceMinor:424 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ab2f241cbd27f93527f68f203a29005c910414f14a4eeedc9e501ddbba116f98/userdata/shm DeviceMajor:0 DeviceMinor:81 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-136 DeviceMajor:0 DeviceMinor:136 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ecbb798a-2024-4984-89c1-56ad55912e77/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:197 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/42b2d2d6-78f0-457a-b182-09dd65dc3372/volumes/kubernetes.io~projected/kube-api-access-jtds7 DeviceMajor:0 DeviceMinor:257 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/80ff3b20e76a7e6edd5a96faa8e3e9373e5b4fd8d417efa5e26120fc44e52d74/userdata/shm DeviceMajor:0 DeviceMinor:266 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-307 DeviceMajor:0 DeviceMinor:307 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/1f359854-4031-4cf7-9ef4-877e4a90d249/volumes/kubernetes.io~secret/console-oauth-config DeviceMajor:0 DeviceMinor:349 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/263a55c9-0e5a-47bc-bec1-20cd01e12942/volumes/kubernetes.io~secret/secret-metrics-server-client-certs DeviceMajor:0 DeviceMinor:361 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:201 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1f359854-4031-4cf7-9ef4-877e4a90d249/volumes/kubernetes.io~secret/console-serving-cert DeviceMajor:0 DeviceMinor:350 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-368 DeviceMajor:0 DeviceMinor:368 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/263a55c9-0e5a-47bc-bec1-20cd01e12942/volumes/kubernetes.io~secret/secret-metrics-server-tls DeviceMajor:0 DeviceMinor:362 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/secret-alertmanager-kube-rbac-proxy-metric DeviceMajor:0 DeviceMinor:301 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~empty-dir/config-out DeviceMajor:0 DeviceMinor:305 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~projected/kube-api-access-8brjr DeviceMajor:0 DeviceMinor:309 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1b28ccf8718b1b23df3e6129770c007aef7bbf94cb020f329fa3135fbe247c8d/userdata/shm DeviceMajor:0 DeviceMinor:343 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-370 DeviceMajor:0 DeviceMinor:370 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/b67fee5e-0972-4cfc-987d-b0ae79e6589d/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:447 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f8e3edc927fdc1674f512ef088d22ea4aebb6823380e08537e85cf60933eec98/userdata/shm DeviceMajor:0 DeviceMinor:452 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9822bfc7-212f-4cf6-8d56-eb44c58c359b/volumes/kubernetes.io~projected/kube-api-access-nck5c DeviceMajor:0 DeviceMinor:53 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/11550a43-fc10-43e9-835c-ae7c8b21be03/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:50 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-217 DeviceMajor:0 DeviceMinor:217 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-315 DeviceMajor:0 DeviceMinor:315 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-468 DeviceMajor:0 DeviceMinor:468 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-246 DeviceMajor:0 DeviceMinor:246 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/cluster-tls-config DeviceMajor:0 DeviceMinor:295 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~projected/kube-api-access-z2t26 DeviceMajor:0 DeviceMinor:333 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-371 DeviceMajor:0 DeviceMinor:371 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-378 DeviceMajor:0 DeviceMinor:378 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/11550a43-fc10-43e9-835c-ae7c8b21be03/volumes/kubernetes.io~projected/kube-api-access-zdzp8 DeviceMajor:0 DeviceMinor:58 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-226 DeviceMajor:0 DeviceMinor:226 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0e786874-ce56-46f8-87f5-920dacacd9fc/volumes/kubernetes.io~secret/kube-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:263 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~secret/web-config DeviceMajor:0 DeviceMinor:304 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/3acfed5c-f071-4450-9e72-d923cc492d3f/volumes/kubernetes.io~projected/kube-api-access-btbrv DeviceMajor:0 DeviceMinor:356 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-420 DeviceMajor:0 DeviceMinor:420 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/652a5a58-ad99-49bb-b371-743c2816482d/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:48 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/17bf4ea6d518f6a340d2f3cc1e666c8fd718ff015e8c61fb7cb9ac843d44273c/userdata/shm DeviceMajor:0 DeviceMinor:80 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-326 DeviceMajor:0 DeviceMinor:326 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-metrics DeviceMajor:0 DeviceMinor:342 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/1f359854-4031-4cf7-9ef4-877e4a90d249/volumes/kubernetes.io~projected/kube-api-access-vsptj DeviceMajor:0 DeviceMinor:351 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-354 DeviceMajor:0 DeviceMinor:354 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-78 DeviceMajor:0 DeviceMinor:78 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/42b2d2d6-78f0-457a-b182-09dd65dc3372/volumes/kubernetes.io~secret/openshift-state-metrics-tls DeviceMajor:0 DeviceMinor:251 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-287 DeviceMajor:0 DeviceMinor:287 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-grpc-tls DeviceMajor:0 DeviceMinor:341 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-345 DeviceMajor:0 DeviceMinor:345 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-388 DeviceMajor:0 DeviceMinor:388 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volume-subpaths/cluster-tls-config/alertmanager/10 DeviceMajor:0 DeviceMinor:295 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0143f87b71fb5861a7461820eb8d145a3d7692cf0dc40802b3fbc3efe002ded1/userdata/shm DeviceMajor:0 DeviceMinor:224 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-399 DeviceMajor:0 DeviceMinor:399 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-404 DeviceMajor:0 DeviceMinor:404 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-415 DeviceMajor:0 DeviceMinor:415 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-458 DeviceMajor:0 DeviceMinor:458 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-460 DeviceMajor:0 DeviceMinor:460 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cda0bc7c-777f-4965-b097-6b5f3b6bfc52/volumes/kubernetes.io~projected/kube-api-access-77jh2 DeviceMajor:0 DeviceMinor:56 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ecbb798a-2024-4984-89c1-56ad55912e77/volumes/kubernetes.io~projected/kube-api-access-fsb2s DeviceMajor:0 DeviceMinor:204 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/0e7ff7bd6e351621e118ef488ba19ec4e7a6c27eb2d14ae71d3350f49d7fbbc0/userdata/shm DeviceMajor:0 DeviceMinor:205 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-235 DeviceMajor:0 DeviceMinor:235 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f2f6cee433b8f8ac45f4eaedafa95e02bc0cc4cfb7d070615acc3657d05f8d62/userdata/shm DeviceMajor:0 DeviceMinor:258 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy-rules DeviceMajor:0 DeviceMinor:337 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f55989fc59f099067d253917e0f1844949c4e949832e54d51fe440249fa95219/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-228 DeviceMajor:0 DeviceMinor:228 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-272 DeviceMajor:0 DeviceMinor:272 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-364 DeviceMajor:0 DeviceMinor:364 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-383 DeviceMajor:0 DeviceMinor:383 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a6c94923-1962-425d-8e53-ad523e78619b/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:47 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4411f88293ac8985be9ff0bf0c171ca23852e5509dc762a19cce13c841ae8c4d/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/652a5a58-ad99-49bb-b371-743c2816482d/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/0e786874-ce56-46f8-87f5-920dacacd9fc/volumes/kubernetes.io~secret/kube-state-metrics-tls DeviceMajor:0 DeviceMinor:262 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/cc73f4b8-ee7e-4915-a883-e26fa87a89db/volumes/kubernetes.io~projected/kube-api-access-hg9z4 DeviceMajor:0 DeviceMinor:51 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/de7265384d17948ad084102cbb4ca2c97b44c67b5a062aba3e6c16d511b8d2a1/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-kube-rbac-proxy DeviceMajor:0 DeviceMinor:339 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-213 DeviceMajor:0 DeviceMinor:213 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-230 DeviceMajor:0 DeviceMinor:230 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/42b2d2d6-78f0-457a-b182-09dd65dc3372/volumes/kubernetes.io~secret/openshift-state-metrics-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:256 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ca586ba2-2d05-45f8-aab9-127b93ea1c60/volumes/kubernetes.io~secret/node-exporter-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:260 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/9ae72d3a-8b68-4d30-93d7-f03418559928/volumes/kubernetes.io~secret/secret-thanos-querier-tls DeviceMajor:0 DeviceMinor:340 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-347 DeviceMajor:0 DeviceMinor:347 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes/kubernetes.io~projected/tls-assets DeviceMajor:0 DeviceMinor:306 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-310 DeviceMajor:0 DeviceMinor:310 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-456 DeviceMajor:0 DeviceMinor:456 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/aad11b046a3a03af247ae3e4b2047008c22032eb02ce4ad94ea40e7b52a05268/userdata/shm DeviceMajor:0 DeviceMinor:209 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/run/containers/storage/overlay-containers/3725fc11d97fb1eeafa752dfadf0018bef6aeedbde6ca23e8d8b731fb7f0f2d4/userdata/shm DeviceMajor:0 DeviceMinor:454 Capacity:67108864 Type:vfs Inodes:4048381 HasInodes:true} {Device:/var/lib/kubelet/pods/ea684836-96a5-4d68-a414-c3db5de7564e/volumes/kubernetes.io~projected/kube-api-access-ghc57 DeviceMajor:0 DeviceMinor:54 Capacity:31985737728 Type:vfs Inodes:4048381 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-241 DeviceMajor:0 DeviceMinor:241 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:0143f87b71fb586 MacAddress:d2:cc:ac:c1:5d:8d Speed:10000 Mtu:8901} {Name:0e7ff7bd6e35162 MacAddress:7a:ac:d3:e8:f2:44 Speed:10000 Mtu:8901} {Name:1b28ccf8718b1b2 MacAddress:f6:e5:aa:43:d4:f7 Speed:10000 Mtu:8901} {Name:3725fc11d97fb1e MacAddress:d6:1e:10:97:01:c6 Speed:10000 Mtu:8901} {Name:6b05236719987d7 MacAddress:16:54:75:50:a8:a3 Speed:10000 Mtu:8901} {Name:7bc50e16469aee5 MacAddress:ee:d4:e1:21:24:b2 Speed:10000 Mtu:8901} {Name:8dafede46b3521e MacAddress:12:a3:0c:c2:12:06 Speed:10000 Mtu:8901} {Name:a5dcaf13f72dae8 MacAddress:26:10:04:9a:85:7b Speed:10000 Mtu:8901} {Name:aad11b046a3a03a MacAddress:16:81:7e:08:58:74 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:06:ff:f9:9e:44:11 Speed:0 Mtu:9001} {Name:br-int MacAddress:fe:dc:83:16:51:44 Speed:0 Mtu:8901} {Name:ens5 MacAddress:06:ff:f9:9e:44:11 Speed:0 Mtu:9001} {Name:f2f6cee433b8f8a MacAddress:6e:f7:75:b7:ff:56 Speed:10000 Mtu:8901} {Name:f8e3edc927fdc16 MacAddress:4a:22:77:4b:60:45 Speed:10000 Mtu:8901} {Name:fc2294e8c03f9d3 MacAddress:b6:f5:18:21:36:cd Speed:10000 Mtu:8901} {Name:fe08928888a95c7 MacAddress:fe:70:d2:42:3f:f5 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:36:96:e8:8c:22:4c Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:86:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:06:16:b0:c3:a6:84 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164337152 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.128374    5704 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.128420    5704 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.128695    5704 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.128720    5704 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-33.us-west-2.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.128833    5704 topology_manager.go:143] "Creating topology manager with none policy"
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.128841    5704 container_manager_linux.go:311] "Creating device plugin manager"
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.128851    5704 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.128863    5704 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.128932    5704 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager"
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.129097    5704 state_mem.go:41] "Initialized" logger="CPUManager state memory"
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.129195    5704 server.go:1235] "Using root directory" path="/var/lib/kubelet"
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.129226    5704 kubelet.go:493] "Attempting to sync node with API server"
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.129239    5704 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests"
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.129249    5704 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.129257    5704 kubelet.go:405] "Adding apiserver pod source"
Oct 30 18:47:07.129732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.129265    5704 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Oct 30 18:47:07.130535 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.129985    5704 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:47:07.130535 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.129998    5704 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:47:07.130535 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.130500    5704 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1"
Oct 30 18:47:07.133122 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133024    5704 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint"
Oct 30 18:47:07.133122 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133057    5704 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Oct 30 18:47:07.133264 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133162    5704 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem"
Oct 30 18:47:07.133403 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133387    5704 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Oct 30 18:47:07.133469 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133429    5704 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Oct 30 18:47:07.133579 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133567    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Oct 30 18:47:07.133635 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133586    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Oct 30 18:47:07.133635 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133597    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Oct 30 18:47:07.133635 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133605    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Oct 30 18:47:07.133635 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133614    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Oct 30 18:47:07.133635 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133624    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Oct 30 18:47:07.133635 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133633    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Oct 30 18:47:07.133921 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133642    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Oct 30 18:47:07.133921 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133654    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Oct 30 18:47:07.133921 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133663    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Oct 30 18:47:07.133921 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133676    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Oct 30 18:47:07.133921 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133690    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Oct 30 18:47:07.133921 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133726    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Oct 30 18:47:07.133921 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.133737    5704 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Oct 30 18:47:07.137020 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.137003    5704 server.go:1263] "Started kubelet"
Oct 30 18:47:07.137753 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.137258    5704 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Oct 30 18:47:07.137753 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.137630    5704 server_v1.go:49] "podresources" method="list" useActivePods=true
Oct 30 18:47:07.137559 ip-10-0-139-33 systemd[1]: Started Kubernetes Kubelet.
Oct 30 18:47:07.137968 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.137941    5704 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Oct 30 18:47:07.138790 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.137094    5704 server.go:182] "Starting to listen" address="0.0.0.0" port=10250
Oct 30 18:47:07.142474 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.141491    5704 server.go:317] "Adding debug handlers to kubelet server"
Oct 30 18:47:07.144631 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.144614    5704 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer"
Oct 30 18:47:07.144839 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.144813    5704 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Oct 30 18:47:07.146817 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:07.146798    5704 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-33.us-west-2.compute.internal\" not found"
Oct 30 18:47:07.146902 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.146834    5704 volume_manager.go:309] "The desired_state_of_world populator starts"
Oct 30 18:47:07.146902 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.146841    5704 volume_manager.go:311] "Starting Kubelet Volume Manager"
Oct 30 18:47:07.147026 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.147012    5704 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Oct 30 18:47:07.147679 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.147651    5704 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:40:57 +0000 UTC" deadline="2028-08-13 04:55:42.194060514 +0000 UTC"
Oct 30 18:47:07.147753 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.147680    5704 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15658h8m35.046386148s"
Oct 30 18:47:07.147916 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.147901    5704 factory.go:55] Registering systemd factory
Oct 30 18:47:07.147991 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.147918    5704 factory.go:223] Registration of the systemd container factory successfully
Oct 30 18:47:07.148228 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.148213    5704 factory.go:153] Registering CRI-O factory
Oct 30 18:47:07.148308 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.148231    5704 factory.go:223] Registration of the crio container factory successfully
Oct 30 18:47:07.148308 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.148290    5704 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Oct 30 18:47:07.148308 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.148308    5704 factory.go:103] Registering Raw factory
Oct 30 18:47:07.148500 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.148322    5704 manager.go:1199] Started watching for new ooms in manager
Oct 30 18:47:07.148745 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.148730    5704 manager.go:319] Starting recovery of all containers
Oct 30 18:47:07.149406 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149371    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ea684836-96a5-4d68-a414-c3db5de7564e" volumeName="kubernetes.io/configmap/ea684836-96a5-4d68-a414-c3db5de7564e-iptables-alerter-script" seLinuxMountContext=""
Oct 30 18:47:07.149493 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149410    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="263a55c9-0e5a-47bc-bec1-20cd01e12942" volumeName="kubernetes.io/empty-dir/263a55c9-0e5a-47bc-bec1-20cd01e12942-audit-log" seLinuxMountContext=""
Oct 30 18:47:07.149493 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149423    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2717f98e-b200-43c5-a089-73fb6a522933" volumeName="kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-main-db" seLinuxMountContext=""
Oct 30 18:47:07.149493 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149448    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" volumeName="kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:47:07.149493 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149461    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" volumeName="kubernetes.io/projected/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-kube-api-access-pvkcs" seLinuxMountContext=""
Oct 30 18:47:07.149493 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149472    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="652a5a58-ad99-49bb-b371-743c2816482d" volumeName="kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-ovnkube-script-lib" seLinuxMountContext=""
Oct 30 18:47:07.149493 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149492    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ae72d3a-8b68-4d30-93d7-f03418559928" volumeName="kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-tls" seLinuxMountContext=""
Oct 30 18:47:07.149764 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149505    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="b67fee5e-0972-4cfc-987d-b0ae79e6589d" volumeName="kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret" seLinuxMountContext=""
Oct 30 18:47:07.149764 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149519    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca586ba2-2d05-45f8-aab9-127b93ea1c60" volumeName="kubernetes.io/secret/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 18:47:07.149764 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149543    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ae72d3a-8b68-4d30-93d7-f03418559928" volumeName="kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-grpc-tls" seLinuxMountContext=""
Oct 30 18:47:07.149764 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149590    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca586ba2-2d05-45f8-aab9-127b93ea1c60" volumeName="kubernetes.io/configmap/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-accelerators-collector-config" seLinuxMountContext=""
Oct 30 18:47:07.149764 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149640    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2717f98e-b200-43c5-a089-73fb6a522933" volumeName="kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-metric" seLinuxMountContext=""
Oct 30 18:47:07.149764 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149691    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="652a5a58-ad99-49bb-b371-743c2816482d" volumeName="kubernetes.io/projected/652a5a58-ad99-49bb-b371-743c2816482d-kube-api-access-r2g7p" seLinuxMountContext=""
Oct 30 18:47:07.149764 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149707    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ae72d3a-8b68-4d30-93d7-f03418559928" volumeName="kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-metrics" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149751    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca586ba2-2d05-45f8-aab9-127b93ea1c60" volumeName="kubernetes.io/configmap/ca586ba2-2d05-45f8-aab9-127b93ea1c60-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149790    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ea684836-96a5-4d68-a414-c3db5de7564e" volumeName="kubernetes.io/projected/ea684836-96a5-4d68-a414-c3db5de7564e-kube-api-access-ghc57" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149802    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2717f98e-b200-43c5-a089-73fb6a522933" volumeName="kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149813    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2717f98e-b200-43c5-a089-73fb6a522933" volumeName="kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-config-volume" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149851    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" volumeName="kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-whereabouts-flatfile-configmap" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149866    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d07d694-4e2c-401a-aeab-0c9c4f530e87" volumeName="kubernetes.io/projected/3d07d694-4e2c-401a-aeab-0c9c4f530e87-kube-api-access-bp6fz" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149877    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc73f4b8-ee7e-4915-a883-e26fa87a89db" volumeName="kubernetes.io/configmap/cc73f4b8-ee7e-4915-a883-e26fa87a89db-cni-binary-copy" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149922    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ecbb798a-2024-4984-89c1-56ad55912e77" volumeName="kubernetes.io/secret/ecbb798a-2024-4984-89c1-56ad55912e77-insights-runtime-extractor-tls" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149935    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11550a43-fc10-43e9-835c-ae7c8b21be03" volumeName="kubernetes.io/empty-dir/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-tuned" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149946    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f359854-4031-4cf7-9ef4-877e4a90d249" volumeName="kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149984    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3acfed5c-f071-4450-9e72-d923cc492d3f" volumeName="kubernetes.io/projected/3acfed5c-f071-4450-9e72-d923cc492d3f-kube-api-access-btbrv" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.149996    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="738e26c2-0aac-4df0-961d-8c7404c8d6ab" volumeName="kubernetes.io/empty-dir/738e26c2-0aac-4df0-961d-8c7404c8d6ab-tmp-dir" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150018    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ae72d3a-8b68-4d30-93d7-f03418559928" volumeName="kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-rules" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150079    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca586ba2-2d05-45f8-aab9-127b93ea1c60" volumeName="kubernetes.io/projected/ca586ba2-2d05-45f8-aab9-127b93ea1c60-kube-api-access-wfhmw" seLinuxMountContext=""
Oct 30 18:47:07.150094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150100    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f359854-4031-4cf7-9ef4-877e4a90d249" volumeName="kubernetes.io/empty-dir/1f359854-4031-4cf7-9ef4-877e4a90d249-tmp" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150111    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc73f4b8-ee7e-4915-a883-e26fa87a89db" volumeName="kubernetes.io/configmap/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-daemon-config" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150133    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ecbb798a-2024-4984-89c1-56ad55912e77" volumeName="kubernetes.io/configmap/ecbb798a-2024-4984-89c1-56ad55912e77-kube-rbac-proxy-cm" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150155    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e786874-ce56-46f8-87f5-920dacacd9fc" volumeName="kubernetes.io/projected/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-api-access-mqxts" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150166    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e786874-ce56-46f8-87f5-920dacacd9fc" volumeName="kubernetes.io/secret/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-tls" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150180    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11550a43-fc10-43e9-835c-ae7c8b21be03" volumeName="kubernetes.io/projected/11550a43-fc10-43e9-835c-ae7c8b21be03-kube-api-access-zdzp8" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150210    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="263a55c9-0e5a-47bc-bec1-20cd01e12942" volumeName="kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-secret-metrics-server-client-certs" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150226    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" volumeName="kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cni-sysctl-allowlist" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150238    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6" volumeName="kubernetes.io/secret/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6-cert" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150249    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cc73f4b8-ee7e-4915-a883-e26fa87a89db" volumeName="kubernetes.io/projected/cc73f4b8-ee7e-4915-a883-e26fa87a89db-kube-api-access-hg9z4" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150260    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2717f98e-b200-43c5-a089-73fb6a522933" volumeName="kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150272    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2717f98e-b200-43c5-a089-73fb6a522933" volumeName="kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-tls-assets" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150283    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ecbb798a-2024-4984-89c1-56ad55912e77" volumeName="kubernetes.io/empty-dir/ecbb798a-2024-4984-89c1-56ad55912e77-data-volume" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150294    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e786874-ce56-46f8-87f5-920dacacd9fc" volumeName="kubernetes.io/secret/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150305    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ddbd05bb-e311-4a16-af9e-0ab5fa5103c0" volumeName="kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150317    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f359854-4031-4cf7-9ef4-877e4a90d249" volumeName="kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150327    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f359854-4031-4cf7-9ef4-877e4a90d249" volumeName="kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150339    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2717f98e-b200-43c5-a089-73fb6a522933" volumeName="kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-trusted-ca-bundle" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150349    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2717f98e-b200-43c5-a089-73fb6a522933" volumeName="kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-config-out" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150362    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42b2d2d6-78f0-457a-b182-09dd65dc3372" volumeName="kubernetes.io/projected/42b2d2d6-78f0-457a-b182-09dd65dc3372-kube-api-access-jtds7" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150374    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42b2d2d6-78f0-457a-b182-09dd65dc3372" volumeName="kubernetes.io/secret/42b2d2d6-78f0-457a-b182-09dd65dc3372-openshift-state-metrics-kube-rbac-proxy-config" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150386    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4cb022e6-43bd-4d71-9935-f0e91130735b" volumeName="kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150398    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9822bfc7-212f-4cf6-8d56-eb44c58c359b" volumeName="kubernetes.io/configmap/9822bfc7-212f-4cf6-8d56-eb44c58c359b-serviceca" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150410    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e786874-ce56-46f8-87f5-920dacacd9fc" volumeName="kubernetes.io/empty-dir/0e786874-ce56-46f8-87f5-920dacacd9fc-volume-directive-shadow" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150422    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f359854-4031-4cf7-9ef4-877e4a90d249" volumeName="kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150435    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2717f98e-b200-43c5-a089-73fb6a522933" volumeName="kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-web-config" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150447    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42b2d2d6-78f0-457a-b182-09dd65dc3372" volumeName="kubernetes.io/configmap/42b2d2d6-78f0-457a-b182-09dd65dc3372-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150458    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9822bfc7-212f-4cf6-8d56-eb44c58c359b" volumeName="kubernetes.io/projected/9822bfc7-212f-4cf6-8d56-eb44c58c359b-kube-api-access-nck5c" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150501    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca586ba2-2d05-45f8-aab9-127b93ea1c60" volumeName="kubernetes.io/empty-dir/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-textfile" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150514    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f359854-4031-4cf7-9ef4-877e4a90d249" volumeName="kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150526    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="263a55c9-0e5a-47bc-bec1-20cd01e12942" volumeName="kubernetes.io/configmap/263a55c9-0e5a-47bc-bec1-20cd01e12942-metrics-server-audit-profiles" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150538    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2717f98e-b200-43c5-a089-73fb6a522933" volumeName="kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-cluster-tls-config" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150550    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="42b2d2d6-78f0-457a-b182-09dd65dc3372" volumeName="kubernetes.io/secret/42b2d2d6-78f0-457a-b182-09dd65dc3372-openshift-state-metrics-tls" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150563    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6c94923-1962-425d-8e53-ad523e78619b" volumeName="kubernetes.io/secret/a6c94923-1962-425d-8e53-ad523e78619b-agent-certs" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150576    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ecbb798a-2024-4984-89c1-56ad55912e77" volumeName="kubernetes.io/projected/ecbb798a-2024-4984-89c1-56ad55912e77-kube-api-access-fsb2s" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150597    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e786874-ce56-46f8-87f5-920dacacd9fc" volumeName="kubernetes.io/configmap/0e786874-ce56-46f8-87f5-920dacacd9fc-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150609    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="11550a43-fc10-43e9-835c-ae7c8b21be03" volumeName="kubernetes.io/empty-dir/11550a43-fc10-43e9-835c-ae7c8b21be03-tmp" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150625    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f359854-4031-4cf7-9ef4-877e4a90d249" volumeName="kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150638    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="263a55c9-0e5a-47bc-bec1-20cd01e12942" volumeName="kubernetes.io/projected/263a55c9-0e5a-47bc-bec1-20cd01e12942-kube-api-access-t9cjq" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150650    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2717f98e-b200-43c5-a089-73fb6a522933" volumeName="kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-main-tls" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150661    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a6c94923-1962-425d-8e53-ad523e78619b" volumeName="kubernetes.io/configmap/a6c94923-1962-425d-8e53-ad523e78619b-konnectivity-ca" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150677    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cda0bc7c-777f-4965-b097-6b5f3b6bfc52" volumeName="kubernetes.io/projected/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-kube-api-access-77jh2" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150690    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0e786874-ce56-46f8-87f5-920dacacd9fc" volumeName="kubernetes.io/configmap/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-custom-resource-state-configmap" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150705    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="263a55c9-0e5a-47bc-bec1-20cd01e12942" volumeName="kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-client-ca-bundle" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150719    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2717f98e-b200-43c5-a089-73fb6a522933" volumeName="kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-kube-api-access-8brjr" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150733    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3acfed5c-f071-4450-9e72-d923cc492d3f" volumeName="kubernetes.io/secret/3acfed5c-f071-4450-9e72-d923cc492d3f-monitoring-plugin-cert" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150747    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d07d694-4e2c-401a-aeab-0c9c4f530e87" volumeName="kubernetes.io/configmap/3d07d694-4e2c-401a-aeab-0c9c4f530e87-config-volume" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150761    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d07d694-4e2c-401a-aeab-0c9c4f530e87" volumeName="kubernetes.io/secret/3d07d694-4e2c-401a-aeab-0c9c4f530e87-metrics-tls" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150774    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="652a5a58-ad99-49bb-b371-743c2816482d" volumeName="kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-env-overrides" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150792    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="738e26c2-0aac-4df0-961d-8c7404c8d6ab" volumeName="kubernetes.io/projected/738e26c2-0aac-4df0-961d-8c7404c8d6ab-kube-api-access-w6fjw" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150806    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="263a55c9-0e5a-47bc-bec1-20cd01e12942" volumeName="kubernetes.io/configmap/263a55c9-0e5a-47bc-bec1-20cd01e12942-configmap-kubelet-serving-ca-bundle" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150818    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="263a55c9-0e5a-47bc-bec1-20cd01e12942" volumeName="kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-secret-metrics-server-tls" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150830    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="652a5a58-ad99-49bb-b371-743c2816482d" volumeName="kubernetes.io/secret/652a5a58-ad99-49bb-b371-743c2816482d-ovn-node-metrics-cert" seLinuxMountContext=""
Oct 30 18:47:07.150826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150844    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6" volumeName="kubernetes.io/projected/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6-kube-api-access-z2gh2" seLinuxMountContext=""
Oct 30 18:47:07.153405 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150857    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ae72d3a-8b68-4d30-93d7-f03418559928" volumeName="kubernetes.io/configmap/9ae72d3a-8b68-4d30-93d7-f03418559928-metrics-client-ca" seLinuxMountContext=""
Oct 30 18:47:07.153405 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150868    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ae72d3a-8b68-4d30-93d7-f03418559928" volumeName="kubernetes.io/empty-dir/9ae72d3a-8b68-4d30-93d7-f03418559928-tmp" seLinuxMountContext=""
Oct 30 18:47:07.153405 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150881    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ae72d3a-8b68-4d30-93d7-f03418559928" volumeName="kubernetes.io/projected/9ae72d3a-8b68-4d30-93d7-f03418559928-kube-api-access-z2t26" seLinuxMountContext=""
Oct 30 18:47:07.153405 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150892    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ae72d3a-8b68-4d30-93d7-f03418559928" volumeName="kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 18:47:07.153405 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150906    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="1f359854-4031-4cf7-9ef4-877e4a90d249" volumeName="kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca" seLinuxMountContext=""
Oct 30 18:47:07.153405 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150921    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2717f98e-b200-43c5-a089-73fb6a522933" volumeName="kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-web" seLinuxMountContext=""
Oct 30 18:47:07.153405 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150937    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d07d694-4e2c-401a-aeab-0c9c4f530e87" volumeName="kubernetes.io/empty-dir/3d07d694-4e2c-401a-aeab-0c9c4f530e87-tmp-dir" seLinuxMountContext=""
Oct 30 18:47:07.153405 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150948    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4cb022e6-43bd-4d71-9935-f0e91130735b" volumeName="kubernetes.io/projected/4cb022e6-43bd-4d71-9935-f0e91130735b-kube-api-access-vrrbp" seLinuxMountContext=""
Oct 30 18:47:07.153405 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150960    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="652a5a58-ad99-49bb-b371-743c2816482d" volumeName="kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-ovnkube-config" seLinuxMountContext=""
Oct 30 18:47:07.153405 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150973    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9ae72d3a-8b68-4d30-93d7-f03418559928" volumeName="kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy" seLinuxMountContext=""
Oct 30 18:47:07.153405 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150985    5704 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ca586ba2-2d05-45f8-aab9-127b93ea1c60" volumeName="kubernetes.io/secret/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-tls" seLinuxMountContext=""
Oct 30 18:47:07.153405 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.150997    5704 reconstruct.go:97] "Volume reconstruction finished"
Oct 30 18:47:07.153405 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.151005    5704 reconciler.go:29] "Reconciler: start to sync state"
Oct 30 18:47:07.170189 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.170162    5704 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Oct 30 18:47:07.171506 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.171487    5704 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Oct 30 18:47:07.171506 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.171509    5704 status_manager.go:249] "Starting to sync pod status with apiserver"
Oct 30 18:47:07.171664 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.171526    5704 kubelet.go:2521] "Starting kubelet main sync loop"
Oct 30 18:47:07.171664 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:07.171562    5704 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Oct 30 18:47:07.186485 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.186453    5704 generic.go:356] "Generic (PLEG): container finished" podID="91959f1cc4ebe49096831f4c0e77120d" containerID="4cc911b2e82c35b1d86d640e48724dd1eabdab4039c18a15873a82f13a5e0954" exitCode=0
Oct 30 18:47:07.209180 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.209153    5704 generic.go:356] "Generic (PLEG): container finished" podID="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" containerID="a80f5bf5a29d3c9c2b28aa8a585c4ada1b321f1fd6908764cd22f3b516a168ab" exitCode=0
Oct 30 18:47:07.209180 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.209182    5704 generic.go:356] "Generic (PLEG): container finished" podID="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" containerID="8d3a9f398d2d5c1829e4f8bd962541214dd0581dc0222273335381945cb0e43d" exitCode=0
Oct 30 18:47:07.209316 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.209192    5704 generic.go:356] "Generic (PLEG): container finished" podID="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" containerID="04b7e5c487388afec471001bac792c6f9d78f95a7241064fe8cde1874e90ac58" exitCode=0
Oct 30 18:47:07.209316 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.209199    5704 generic.go:356] "Generic (PLEG): container finished" podID="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" containerID="92cc3dbffa14f0a3704766ad16b246d88e3772fa111bd7a657f64abc02949c6c" exitCode=0
Oct 30 18:47:07.209316 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.209207    5704 generic.go:356] "Generic (PLEG): container finished" podID="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" containerID="fa01b73e715d03a0f9fce5e59a379833c404fdedf4cded85d09cb898646eec63" exitCode=0
Oct 30 18:47:07.209316 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.209212    5704 generic.go:356] "Generic (PLEG): container finished" podID="3cbb0cdd-8ae2-4149-84d8-5df55b50e54a" containerID="7466e3999e642dbc919bd1c8fa65e533920be782f3bfb63b017308859b8f1336" exitCode=0
Oct 30 18:47:07.216273 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.216251    5704 generic.go:356] "Generic (PLEG): container finished" podID="2717f98e-b200-43c5-a089-73fb6a522933" containerID="070fcf583e97a5546d1b72cba3246e572a54913e0c473fbbb39dd8e861eaa71f" exitCode=0
Oct 30 18:47:07.230487 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.230465    5704 generic.go:356] "Generic (PLEG): container finished" podID="ca586ba2-2d05-45f8-aab9-127b93ea1c60" containerID="78104c5c61a780315a5419e8b4852b5182dcbb7f7960dd9dc3fedae7dcde499e" exitCode=0
Oct 30 18:47:07.246861 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:07.246843    5704 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-139-33.us-west-2.compute.internal\" not found"
Oct 30 18:47:07.257885 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.257844    5704 manager.go:324] Recovery completed
Oct 30 18:47:07.272155 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:07.272135    5704 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet"
Oct 30 18:47:07.277409 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.277397    5704 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:47:07.279316 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.279302    5704 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:47:07.279371 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.279332    5704 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:47:07.279371 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.279345    5704 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:47:07.281959 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.281940    5704 cpu_manager.go:226] "Starting" policy="none"
Oct 30 18:47:07.281959 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.281957    5704 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:47:07.282096 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.281956    5704 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s"
Oct 30 18:47:07.282096 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.282074    5704 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory"
Oct 30 18:47:07.282212 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.282199    5704 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet=""
Oct 30 18:47:07.282253 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.282213    5704 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={}
Oct 30 18:47:07.282253 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.282228    5704 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint"
Oct 30 18:47:07.282253 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.282234    5704 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet=""
Oct 30 18:47:07.282253 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.282239    5704 policy_none.go:50] "Start"
Oct 30 18:47:07.282253 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.282249    5704 memory_manager.go:187] "Starting memorymanager" policy="None"
Oct 30 18:47:07.282392 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.282258    5704 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Oct 30 18:47:07.282392 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.282379    5704 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint"
Oct 30 18:47:07.282392 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.282387    5704 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint"
Oct 30 18:47:07.282392 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.282392    5704 policy_none.go:44] "Start"
Oct 30 18:47:07.282995 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.282978    5704 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:47:07.283087 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.283006    5704 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:47:07.283087 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.283016    5704 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:47:07.284541 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.284396    5704 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:47:07.284733 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.284711    5704 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:47:07.289071 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.289022    5704 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"}
Oct 30 18:47:07.294843 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.294831    5704 manager.go:341] "Starting Device Plugin manager"
Oct 30 18:47:07.294907 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:07.294853    5704 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Oct 30 18:47:07.294907 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.294870    5704 server.go:87] "Starting device plugin registration server"
Oct 30 18:47:07.295153 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.295142    5704 eviction_manager.go:194] "Eviction manager: starting control loop"
Oct 30 18:47:07.295210 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.295154    5704 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Oct 30 18:47:07.295254 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.295241    5704 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Oct 30 18:47:07.295364 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.295333    5704 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Oct 30 18:47:07.295421 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.295377    5704 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts"
Oct 30 18:47:07.295421 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.295385    5704 plugin_manager.go:121] "Starting Kubelet Plugin Manager"
Oct 30 18:47:07.295942 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.295435    5704 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:47:07.295362478Z","UUID":"a647e18e-a07b-40da-bb4e-5ef0ee8ad113","Handler":null,"Name":"","Endpoint":""}
Oct 30 18:47:07.297096 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:07.297079    5704 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Oct 30 18:47:07.297188 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:07.297113    5704 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-33.us-west-2.compute.internal\" not found"
Oct 30 18:47:07.297257 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.297241    5704 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Oct 30 18:47:07.297299 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.297275    5704 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Oct 30 18:47:07.328834 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.328817    5704 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:47:07.395463 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.395444    5704 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach"
Oct 30 18:47:07.396369 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.396354    5704 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientMemory"
Oct 30 18:47:07.396447 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.396380    5704 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasNoDiskPressure"
Oct 30 18:47:07.396447 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.396390    5704 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-139-33.us-west-2.compute.internal" event="NodeHasSufficientPID"
Oct 30 18:47:07.396447 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.396410    5704 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.420661 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.420641    5704 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161"
Oct 30 18:47:07.420809 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.420796    5704 reconstruct.go:205] "DevicePaths of reconstructed volumes updated"
Oct 30 18:47:07.472232 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.472192    5704 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"]
Oct 30 18:47:07.472811 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.472753    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal" event={"ID":"91959f1cc4ebe49096831f4c0e77120d","Type":"ContainerStarted","Data":"d54e9cef42e70f225c3d005f8da05c9d0ef430eaa57ebb31abd21e9335987b04"}
Oct 30 18:47:07.472910 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.472812    5704 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.472910 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.472891    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal" event={"ID":"91959f1cc4ebe49096831f4c0e77120d","Type":"ContainerDied","Data":"4cc911b2e82c35b1d86d640e48724dd1eabdab4039c18a15873a82f13a5e0954"}
Oct 30 18:47:07.473199 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.472920    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal" event={"ID":"91959f1cc4ebe49096831f4c0e77120d","Type":"ContainerStarted","Data":"daa36dfdd6a1af71a65334aa018896fed1f9e008030b0ff3f29a8bcdf60f895b"}
Oct 30 18:47:07.473199 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.472985    5704 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.473199 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.473012    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal" event={"ID":"9a61da0484b7ed0271380ee2068515a4","Type":"ContainerStarted","Data":"b717f1f59d3109533ef9c11e85f3bb6f50643fe8d4d33c89dd17a814fe4dfb60"}
Oct 30 18:47:07.473199 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.473027    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal" event={"ID":"9a61da0484b7ed0271380ee2068515a4","Type":"ContainerStarted","Data":"f55989fc59f099067d253917e0f1844949c4e949832e54d51fe440249fa95219"}
Oct 30 18:47:07.498474 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.498454    5704 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.498540 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.498504    5704 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.528252 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.528203    5704 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:47:07.528252 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:07.528238    5704 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.528837 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.528823    5704 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:47:07.528869 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:07.528856    5704 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.621635 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.621615    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91959f1cc4ebe49096831f4c0e77120d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"91959f1cc4ebe49096831f4c0e77120d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.621720 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.621642    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91959f1cc4ebe49096831f4c0e77120d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"91959f1cc4ebe49096831f4c0e77120d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.621720 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.621658    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a61da0484b7ed0271380ee2068515a4-config\") pod \"kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"9a61da0484b7ed0271380ee2068515a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.722563 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.722540    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91959f1cc4ebe49096831f4c0e77120d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"91959f1cc4ebe49096831f4c0e77120d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.722644 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.722500    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/91959f1cc4ebe49096831f4c0e77120d-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"91959f1cc4ebe49096831f4c0e77120d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.722644 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.722611    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91959f1cc4ebe49096831f4c0e77120d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"91959f1cc4ebe49096831f4c0e77120d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.722644 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.722630    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a61da0484b7ed0271380ee2068515a4-config\") pod \"kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"9a61da0484b7ed0271380ee2068515a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.722788 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.722664    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9a61da0484b7ed0271380ee2068515a4-config\") pod \"kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"9a61da0484b7ed0271380ee2068515a4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:07.722788 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:07.722692    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91959f1cc4ebe49096831f4c0e77120d-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" (UID: \"91959f1cc4ebe49096831f4c0e77120d\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:08.130358 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.130334    5704 apiserver.go:52] "Watching apiserver"
Oct 30 18:47:08.226850 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.226826    5704 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Oct 30 18:47:08.227598 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.227557    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-j88bw","openshift-insights/insights-runtime-extractor-6d2f8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx","openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc","openshift-network-diagnostics/network-check-target-42cjm","openshift-ingress-canary/ingress-canary-krgch","kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal","openshift-monitoring/node-exporter-qdr6m","kube-system/global-pull-secret-syncer-wpm6z","openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl","openshift-monitoring/alertmanager-main-0","openshift-multus/network-metrics-daemon-pgbfm","openshift-ovn-kubernetes/ovnkube-node-45wjp","openshift-monitoring/thanos-querier-5d769f5588-7flhp","openshift-multus/multus-additional-cni-plugins-ph92c","openshift-console/console-77b7f76bd4-6rfpv","openshift-dns/node-resolver-hx8pq","openshift-image-registry/node-ca-664td","openshift-network-operator/iptables-alerter-v6t5q","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal","openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7","kube-system/konnectivity-agent-nv7ql","openshift-cluster-node-tuning-operator/tuned-jddc7","openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw","openshift-multus/multus-vkvjm"]
Oct 30 18:47:08.242475 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.242454    5704 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:08.242625 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.242581    5704 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:08.249502 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.249481    5704 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Oct 30 18:47:08.277161 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.277140    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Oct 30 18:47:08.277340 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.277326    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Oct 30 18:47:08.277525 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.277511    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Oct 30 18:47:08.278201 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.278183    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-29qlm\""
Oct 30 18:47:08.278251 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.278206    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Oct 30 18:47:08.278340 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.278329    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lj6ts\""
Oct 30 18:47:08.278711 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.278699    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-56wtl\""
Oct 30 18:47:08.279022 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.279012    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Oct 30 18:47:08.279063 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.279024    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Oct 30 18:47:08.279309 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.279293    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Oct 30 18:47:08.279486 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.279475    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Oct 30 18:47:08.279596 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.279584    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Oct 30 18:47:08.279690 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.279677    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Oct 30 18:47:08.279755 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.279680    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Oct 30 18:47:08.279862 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.279841    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-crxtc\""
Oct 30 18:47:08.279862 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.279875    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Oct 30 18:47:08.280027 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.279997    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Oct 30 18:47:08.280313 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.280251    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Oct 30 18:47:08.280436 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.280423    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Oct 30 18:47:08.280722 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.280710    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5tsjg\""
Oct 30 18:47:08.281127 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.281115    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:47:08.281425 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.281408    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Oct 30 18:47:08.281826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.281811    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Oct 30 18:47:08.282144 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.282114    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Oct 30 18:47:08.282240 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.282144    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-q9tbn\""
Oct 30 18:47:08.282240 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.282196    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Oct 30 18:47:08.282240 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.282202    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Oct 30 18:47:08.282240 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.282227    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Oct 30 18:47:08.282451 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.282285    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Oct 30 18:47:08.282582 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.282531    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Oct 30 18:47:08.282582 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.282562    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bpjwm\""
Oct 30 18:47:08.282681 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.282668    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-77vpl\""
Oct 30 18:47:08.282881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.282868    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Oct 30 18:47:08.282946 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.282874    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n4vwp\""
Oct 30 18:47:08.283017 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.283005    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Oct 30 18:47:08.283130 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.283101    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-hgxcg\""
Oct 30 18:47:08.283130 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.283120    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Oct 30 18:47:08.283231 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.283206    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Oct 30 18:47:08.321738 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.321718    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Oct 30 18:47:08.321819 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.321805    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Oct 30 18:47:08.321913 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.321829    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Oct 30 18:47:08.321913 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.321863    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Oct 30 18:47:08.321913 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.321873    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Oct 30 18:47:08.324072 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.324058    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Oct 30 18:47:08.324494 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.324477    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-log-socket\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.324561 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.324510    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cni-binary-copy\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.324561 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.324541    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-modprobe-d\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.324635 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.324588    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysctl-conf\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.324675 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.324642    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-var-lib-kubelet\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.324722 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.324707    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-kubelet\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.324755 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.324739    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-sys\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.324809 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.324792    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.324847 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.324808    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cni-binary-copy\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.324847 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.324836    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b67fee5e-0972-4cfc-987d-b0ae79e6589d-dbus\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:47:08.324916 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.324896    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-host\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.324973 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.324956    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-env-overrides\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.325051 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325017    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.325096 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325079    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Oct 30 18:47:08.325096 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325086    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Oct 30 18:47:08.325170 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325089    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.325170 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325138    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.325260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325165    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/263a55c9-0e5a-47bc-bec1-20cd01e12942-metrics-server-audit-profiles\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.325260 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.325165    5704 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 18:47:08.325260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325196    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-systemd-units\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.325260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325206    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-env-overrides\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.325260 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325223    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-k8s-cni-cncf-io\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.325488 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.325285    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:08.825259353 +0000 UTC m=+1.771427251 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-config" not registered
Oct 30 18:47:08.325488 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325309    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Oct 30 18:47:08.325488 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325329    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Oct 30 18:47:08.325488 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325311    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-accelerators-collector-config\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.325488 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325371    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-config-volume\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.325488 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325396    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.325488 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325385    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Oct 30 18:47:08.325729 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325423    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b67fee5e-0972-4cfc-987d-b0ae79e6589d-kubelet-config\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:47:08.325870 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325852    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/263a55c9-0e5a-47bc-bec1-20cd01e12942-metrics-server-audit-profiles\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.325930 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325912    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.325975 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325955    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ae72d3a-8b68-4d30-93d7-f03418559928-tmp\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.326042 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.325991    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-run-ovn-kubernetes\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.326042 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326016    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-system-cni-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.326145 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326065    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfhmw\" (UniqueName: \"kubernetes.io/projected/ca586ba2-2d05-45f8-aab9-127b93ea1c60-kube-api-access-wfhmw\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.326145 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326097    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9cjq\" (UniqueName: \"kubernetes.io/projected/263a55c9-0e5a-47bc-bec1-20cd01e12942-kube-api-access-t9cjq\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.326145 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326119    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9ae72d3a-8b68-4d30-93d7-f03418559928-tmp\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.326375 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326351    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.326475 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326388    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6-cert\") pod \"ingress-canary-krgch\" (UID: \"8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6\") " pod="openshift-ingress-canary/ingress-canary-krgch"
Oct 30 18:47:08.326535 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326413    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0e786874-ce56-46f8-87f5-920dacacd9fc-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:47:08.326585 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326562    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdzp8\" (UniqueName: \"kubernetes.io/projected/11550a43-fc10-43e9-835c-ae7c8b21be03-kube-api-access-zdzp8\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.326585 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326563    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Oct 30 18:47:08.326745 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326729    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:47:08.326799 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326754    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Oct 30 18:47:08.326799 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326767    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e786874-ce56-46f8-87f5-920dacacd9fc-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:47:08.326898 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326819    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-cnibin\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.326898 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326850    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/0e786874-ce56-46f8-87f5-920dacacd9fc-volume-directive-shadow\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:47:08.326996 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326850    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-socket-dir-parent\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.326996 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.326969    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-etc-kubernetes\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.327098 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327004    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42b2d2d6-78f0-457a-b182-09dd65dc3372-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:47:08.327098 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327018    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Oct 30 18:47:08.327098 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327052    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-netns\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.327232 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327183    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-var-lib-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.327465 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327445    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecbb798a-2024-4984-89c1-56ad55912e77-data-volume\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:47:08.327540 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327464    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cb022e6-43bd-4d71-9935-f0e91130735b-metrics-certs\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:47:08.327540 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327526    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-kubernetes\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.327656 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327570    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/ecbb798a-2024-4984-89c1-56ad55912e77-data-volume\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:47:08.327656 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327576    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a6c94923-1962-425d-8e53-ad523e78619b-konnectivity-ca\") pod \"konnectivity-agent-nv7ql\" (UID: \"a6c94923-1962-425d-8e53-ad523e78619b\") " pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:47:08.327656 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327622    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/42b2d2d6-78f0-457a-b182-09dd65dc3372-metrics-client-ca\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:47:08.327656 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327642    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea684836-96a5-4d68-a414-c3db5de7564e-host-slash\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:47:08.327822 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327766    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btbrv\" (UniqueName: \"kubernetes.io/projected/3acfed5c-f071-4450-9e72-d923cc492d3f-kube-api-access-btbrv\") pod \"monitoring-plugin-77f4f76594-fnkq7\" (UID: \"3acfed5c-f071-4450-9e72-d923cc492d3f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:47:08.327822 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327778    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a6c94923-1962-425d-8e53-ad523e78619b-konnectivity-ca\") pod \"konnectivity-agent-nv7ql\" (UID: \"a6c94923-1962-425d-8e53-ad523e78619b\") " pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:47:08.327923 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327825    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-secret-metrics-server-client-certs\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.328000 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.327981    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-tls\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.328071 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328053    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-etc-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.328135 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328104    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9822bfc7-212f-4cf6-8d56-eb44c58c359b-host\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:47:08.328229 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328209    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-ovn\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.328328 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328254    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-tuned\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.328328 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328281    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d07d694-4e2c-401a-aeab-0c9c4f530e87-metrics-tls\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:47:08.328328 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328311    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-systemd\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.328519 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328337    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a6c94923-1962-425d-8e53-ad523e78619b-agent-certs\") pod \"konnectivity-agent-nv7ql\" (UID: \"a6c94923-1962-425d-8e53-ad523e78619b\") " pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:47:08.328519 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328359    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d07d694-4e2c-401a-aeab-0c9c4f530e87-config-volume\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:47:08.328519 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328378    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.328519 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328411    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77jh2\" (UniqueName: \"kubernetes.io/projected/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-kube-api-access-77jh2\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.328519 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328441    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:47:08.328519 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328473    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ae72d3a-8b68-4d30-93d7-f03418559928-metrics-client-ca\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.328519 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328505    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.328881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328532    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42b2d2d6-78f0-457a-b182-09dd65dc3372-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:47:08.328881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328562    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cnibin\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.328881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328589    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/738e26c2-0aac-4df0-961d-8c7404c8d6ab-hosts-file\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:47:08.328881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328620    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsb2s\" (UniqueName: \"kubernetes.io/projected/ecbb798a-2024-4984-89c1-56ad55912e77-kube-api-access-fsb2s\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:47:08.328881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328650    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:47:08.328881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328672    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-root\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.328881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328699    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.328881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328730    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsptj\" (UniqueName: \"kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.328881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328758    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-secret-metrics-server-tls\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.328881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328788    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-socket-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.328881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328818    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-registration-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.328881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328843    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-etc-selinux\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.328881 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328880    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2gh2\" (UniqueName: \"kubernetes.io/projected/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6-kube-api-access-z2gh2\") pod \"ingress-canary-krgch\" (UID: \"8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6\") " pod="openshift-ingress-canary/ingress-canary-krgch"
Oct 30 18:47:08.329484 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328890    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Oct 30 18:47:08.329484 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328909    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc73f4b8-ee7e-4915-a883-e26fa87a89db-cni-binary-copy\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.329484 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328941    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghc57\" (UniqueName: \"kubernetes.io/projected/ea684836-96a5-4d68-a414-c3db5de7564e-kube-api-access-ghc57\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:47:08.329484 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.328971    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/42b2d2d6-78f0-457a-b182-09dd65dc3372-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:47:08.329484 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329006    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.329484 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329051    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6fjw\" (UniqueName: \"kubernetes.io/projected/738e26c2-0aac-4df0-961d-8c7404c8d6ab-kube-api-access-w6fjw\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:47:08.329484 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329082    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysctl-d\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.329484 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329109    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-tuned\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.329484 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329113    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nck5c\" (UniqueName: \"kubernetes.io/projected/9822bfc7-212f-4cf6-8d56-eb44c58c359b-kube-api-access-nck5c\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:47:08.329894 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329643    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-cni-netd\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.329894 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329692    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/652a5a58-ad99-49bb-b371-743c2816482d-ovn-node-metrics-cert\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.329894 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329720    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-cni-multus\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.329894 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329753    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-conf-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.329894 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329783    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f359854-4031-4cf7-9ef4-877e4a90d249-tmp\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.329894 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329812    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-client-ca-bundle\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.330188 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329924    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-grpc-tls\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.330188 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329959    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-node-log\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.330188 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.329995    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-tls\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.330188 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330026    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-config-out\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.330188 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330077    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 18:47:08.330188 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330077    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-device-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.330450 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330212    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-config-out\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.330450 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330250    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/cc73f4b8-ee7e-4915-a883-e26fa87a89db-cni-binary-copy\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.330450 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330357    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f359854-4031-4cf7-9ef4-877e4a90d249-tmp\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.330450 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330393    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-kubelet\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.330450 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330425    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-sys\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.330682 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330455    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-cni-bin\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.330682 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330494    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2g7p\" (UniqueName: \"kubernetes.io/projected/652a5a58-ad99-49bb-b371-743c2816482d-kube-api-access-r2g7p\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.330682 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330527    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvkcs\" (UniqueName: \"kubernetes.io/projected/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-kube-api-access-pvkcs\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.330682 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330552    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b67fee5e-0972-4cfc-987d-b0ae79e6589d-original-pull-secret\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:47:08.330682 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330560    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11550a43-fc10-43e9-835c-ae7c8b21be03-tmp\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.330682 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330605    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqxts\" (UniqueName: \"kubernetes.io/projected/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-api-access-mqxts\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:47:08.330682 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330633    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/11550a43-fc10-43e9-835c-ae7c8b21be03-tmp\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.330682 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330643    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.330682 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330677    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.331087 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330690    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/652a5a58-ad99-49bb-b371-743c2816482d-ovn-node-metrics-cert\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.331087 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330719    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-ovnkube-script-lib\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.331087 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330745    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/738e26c2-0aac-4df0-961d-8c7404c8d6ab-tmp-dir\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:47:08.331087 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330775    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecbb798a-2024-4984-89c1-56ad55912e77-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:47:08.331087 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.330914    5704 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 18:47:08.331087 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.330972    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/ecbb798a-2024-4984-89c1-56ad55912e77-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:47:08.331087 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.330983    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:08.83096689 +0000 UTC m=+1.777134793 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 18:47:08.331087 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.331013    5704 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 18:47:08.331087 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.331054    5704 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 18:47:08.331087 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.331093    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:08.831071597 +0000 UTC m=+1.777239491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 18:47:08.331433 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331258    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-textfile\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.331433 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331270    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-textfile\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.331433 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331370    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.331433 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.331380    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:08.831360262 +0000 UTC m=+1.777528158 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 18:47:08.331433 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331382    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9ae72d3a-8b68-4d30-93d7-f03418559928-metrics-client-ca\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.331433 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331392    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5229t\" (UniqueName: \"kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t\") pod \"network-check-target-42cjm\" (UID: \"ddbd05bb-e311-4a16-af9e-0ab5fa5103c0\") " pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:47:08.331625 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331437    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ea684836-96a5-4d68-a414-c3db5de7564e-iptables-alerter-script\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:47:08.331625 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331475    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-ovnkube-script-lib\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.331625 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331477    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/738e26c2-0aac-4df0-961d-8c7404c8d6ab-tmp-dir\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:47:08.331625 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331529    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.331625 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331565    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-web-config\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.331625 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331594    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.331857 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331626    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3acfed5c-f071-4450-9e72-d923cc492d3f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-fnkq7\" (UID: \"3acfed5c-f071-4450-9e72-d923cc492d3f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:47:08.331857 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331664    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.331857 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331716    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-cni-bin\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.331857 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331792    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg9z4\" (UniqueName: \"kubernetes.io/projected/cc73f4b8-ee7e-4915-a883-e26fa87a89db-kube-api-access-hg9z4\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.331857 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.331814    5704 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 18:47:08.331857 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331854    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.332239 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331887    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.332239 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331912    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.332239 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331933    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-run\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.332239 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331937    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ea684836-96a5-4d68-a414-c3db5de7564e-iptables-alerter-script\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:47:08.332239 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.331996    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d07d694-4e2c-401a-aeab-0c9c4f530e87-tmp-dir\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:47:08.332239 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.332014    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:08.83197008 +0000 UTC m=+1.778137978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 18:47:08.332239 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332072    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3d07d694-4e2c-401a-aeab-0c9c4f530e87-tmp-dir\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:47:08.332239 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332082    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3d07d694-4e2c-401a-aeab-0c9c4f530e87-config-volume\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:47:08.332239 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332072    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-wtmp\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.332239 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332165    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca586ba2-2d05-45f8-aab9-127b93ea1c60-metrics-client-ca\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.332239 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332211    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/263a55c9-0e5a-47bc-bec1-20cd01e12942-audit-log\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.332727 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332254    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtds7\" (UniqueName: \"kubernetes.io/projected/42b2d2d6-78f0-457a-b182-09dd65dc3372-kube-api-access-jtds7\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:47:08.332727 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332290    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.332727 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332307    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecbb798a-2024-4984-89c1-56ad55912e77-crio-socket\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:47:08.332727 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332357    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecbb798a-2024-4984-89c1-56ad55912e77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:47:08.332727 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332403    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/263a55c9-0e5a-47bc-bec1-20cd01e12942-audit-log\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.333026 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332739    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.333026 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332810    5704 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:47:08.333026 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.332855    5704 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:08.333026 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332869    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ca586ba2-2d05-45f8-aab9-127b93ea1c60-metrics-client-ca\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.333026 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332881    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:47:08.333026 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332952    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.333026 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.332992    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/263a55c9-0e5a-47bc-bec1-20cd01e12942-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.333026 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333013    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-sys-fs\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333088    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333157    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrrbp\" (UniqueName: \"kubernetes.io/projected/4cb022e6-43bd-4d71-9935-f0e91130735b-kube-api-access-vrrbp\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333169    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333179    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333196    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-lib-modules\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333208    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hz6tx\""
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333216    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/263a55c9-0e5a-47bc-bec1-20cd01e12942-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333244    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-ovnkube-config\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333274    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysconfig\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333303    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9822bfc7-212f-4cf6-8d56-eb44c58c359b-serviceca\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333329    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-hostroot\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333382    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333400    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-multus-certs\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333401    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/652a5a58-ad99-49bb-b371-743c2816482d-ovnkube-config\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333417    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.333456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333444    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-run-netns\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333470    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp6fz\" (UniqueName: \"kubernetes.io/projected/3d07d694-4e2c-401a-aeab-0c9c4f530e87-kube-api-access-bp6fz\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333468    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333547    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333626    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333685    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2t26\" (UniqueName: \"kubernetes.io/projected/9ae72d3a-8b68-4d30-93d7-f03418559928-kube-api-access-z2t26\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333731    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-systemd\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333760    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-cni-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333794    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-daemon-config\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333821    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8brjr\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-kube-api-access-8brjr\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333868    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333893    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-slash\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333919    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-system-cni-dir\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.333957    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.333971    5704 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.334000    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-daemon-config\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.334056    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9822bfc7-212f-4cf6-8d56-eb44c58c359b-serviceca\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:47:08.334110 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.334104    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:08.834093362 +0000 UTC m=+1.780261243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"service-ca" not registered
Oct 30 18:47:08.335656 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.335641    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-accelerators-collector-config\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.336512 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.336493    5704 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Oct 30 18:47:08.336584 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.336530    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.336584 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.336541    5704 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-33.us-west-2.compute.internal"
Oct 30 18:47:08.337515 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.337492    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0e786874-ce56-46f8-87f5-920dacacd9fc-metrics-client-ca\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:47:08.342392 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.342375    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.378230 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378207    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Oct 30 18:47:08.378347 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378330    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Oct 30 18:47:08.378397 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378381    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 18:47:08.378442 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378415    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Oct 30 18:47:08.378442 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378433    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Oct 30 18:47:08.378524 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378450    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 18:47:08.378524 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378469    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Oct 30 18:47:08.378601 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378587    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Oct 30 18:47:08.378601 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378595    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-qmmm4\""
Oct 30 18:47:08.378678 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378631    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vjgl9\""
Oct 30 18:47:08.378764 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378752    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Oct 30 18:47:08.378801 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378783    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Oct 30 18:47:08.378878 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378862    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-n4vtv\""
Oct 30 18:47:08.378935 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378917    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Oct 30 18:47:08.378935 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378924    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Oct 30 18:47:08.379005 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378925    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jrxvz\""
Oct 30 18:47:08.379005 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.378945    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 18:47:08.379128 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379084    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 18:47:08.379128 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379087    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Oct 30 18:47:08.379128 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379083    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 18:47:08.379259 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379158    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Oct 30 18:47:08.379363 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379349    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Oct 30 18:47:08.379423 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379399    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Oct 30 18:47:08.379423 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379416    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Oct 30 18:47:08.379491 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379416    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Oct 30 18:47:08.379491 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379449    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Oct 30 18:47:08.379647 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379606    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-uf0a4mc8vfn5\""
Oct 30 18:47:08.379647 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379624    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2kppepquv6fir\""
Oct 30 18:47:08.379802 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379649    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-r47g8\""
Oct 30 18:47:08.379802 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379651    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k86kp\""
Oct 30 18:47:08.379802 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379664    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mgrlf\""
Oct 30 18:47:08.379802 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379674    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-84sx4\""
Oct 30 18:47:08.379802 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379718    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 18:47:08.379802 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379767    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 18:47:08.379802 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379776    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-jd5cp\""
Oct 30 18:47:08.379802 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379789    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-tls\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:47:08.380071 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379909    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-vvlg2\""
Oct 30 18:47:08.380071 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.379991    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Oct 30 18:47:08.380071 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.380060    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/42b2d2d6-78f0-457a-b182-09dd65dc3372-openshift-state-metrics-tls\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:47:08.380158 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.380088    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-client-ca-bundle\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.381178 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.381143    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-secret-metrics-server-tls\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.381219 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.381172    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-grpc-tls\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.382224 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.382211    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-web-config\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.382274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.382258    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-tls\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.382330 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.382275    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.382330 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.382281    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3acfed5c-f071-4450-9e72-d923cc492d3f-monitoring-plugin-cert\") pod \"monitoring-plugin-77f4f76594-fnkq7\" (UID: \"3acfed5c-f071-4450-9e72-d923cc492d3f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:47:08.382330 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.382320    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.382438 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.382388    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.382945 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.382930    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdzp8\" (UniqueName: \"kubernetes.io/projected/11550a43-fc10-43e9-835c-ae7c8b21be03-kube-api-access-zdzp8\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.383121 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.383105    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-btbrv\" (UniqueName: \"kubernetes.io/projected/3acfed5c-f071-4450-9e72-d923cc492d3f-kube-api-access-btbrv\") pod \"monitoring-plugin-77f4f76594-fnkq7\" (UID: \"3acfed5c-f071-4450-9e72-d923cc492d3f\") " pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:47:08.383247 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.383235    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.383284 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.383252    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/ecbb798a-2024-4984-89c1-56ad55912e77-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:47:08.383771 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.383758    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:47:08.383805 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.383765    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.383905 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.383892    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.384998 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.384979    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-kc6rd\""
Oct 30 18:47:08.385302 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.385285    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nck5c\" (UniqueName: \"kubernetes.io/projected/9822bfc7-212f-4cf6-8d56-eb44c58c359b-kube-api-access-nck5c\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:47:08.385372 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.385339    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.385559 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.385538    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsb2s\" (UniqueName: \"kubernetes.io/projected/ecbb798a-2024-4984-89c1-56ad55912e77-kube-api-access-fsb2s\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:47:08.385736 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.385708    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.385822 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.385800    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2gh2\" (UniqueName: \"kubernetes.io/projected/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6-kube-api-access-z2gh2\") pod \"ingress-canary-krgch\" (UID: \"8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6\") " pod="openshift-ingress-canary/ingress-canary-krgch"
Oct 30 18:47:08.386060 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.386028    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghc57\" (UniqueName: \"kubernetes.io/projected/ea684836-96a5-4d68-a414-c3db5de7564e-kube-api-access-ghc57\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:47:08.386137 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.386118    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfhmw\" (UniqueName: \"kubernetes.io/projected/ca586ba2-2d05-45f8-aab9-127b93ea1c60-kube-api-access-wfhmw\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.386196 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.386171    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.386295 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.386278    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-config-volume\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.386340 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.386287    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9cjq\" (UniqueName: \"kubernetes.io/projected/263a55c9-0e5a-47bc-bec1-20cd01e12942-kube-api-access-t9cjq\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.387014 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.386993    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.387086 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.387023    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6-cert\") pod \"ingress-canary-krgch\" (UID: \"8b4bdfd9-e0b1-4d6b-a66e-4647cd0c64f6\") " pod="openshift-ingress-canary/ingress-canary-krgch"
Oct 30 18:47:08.388790 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.388775    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/263a55c9-0e5a-47bc-bec1-20cd01e12942-secret-metrics-server-client-certs\") pod \"metrics-server-85d7bcfb5c-6vmfw\" (UID: \"263a55c9-0e5a-47bc-bec1-20cd01e12942\") " pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:08.388836 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.388787    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/9ae72d3a-8b68-4d30-93d7-f03418559928-secret-thanos-querier-tls\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.389934 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.389917    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/42b2d2d6-78f0-457a-b182-09dd65dc3372-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:47:08.390003 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.389947    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3d07d694-4e2c-401a-aeab-0c9c4f530e87-metrics-tls\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:47:08.390003 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.389961    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a6c94923-1962-425d-8e53-ad523e78619b-agent-certs\") pod \"konnectivity-agent-nv7ql\" (UID: \"a6c94923-1962-425d-8e53-ad523e78619b\") " pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:47:08.393261 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.393241    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvkcs\" (UniqueName: \"kubernetes.io/projected/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-kube-api-access-pvkcs\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.393326 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.393259    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2g7p\" (UniqueName: \"kubernetes.io/projected/652a5a58-ad99-49bb-b371-743c2816482d-kube-api-access-r2g7p\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.393326 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.393269    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77jh2\" (UniqueName: \"kubernetes.io/projected/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-kube-api-access-77jh2\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.393326 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.393290    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg9z4\" (UniqueName: \"kubernetes.io/projected/cc73f4b8-ee7e-4915-a883-e26fa87a89db-kube-api-access-hg9z4\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.393526 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.393512    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqxts\" (UniqueName: \"kubernetes.io/projected/0e786874-ce56-46f8-87f5-920dacacd9fc-kube-api-access-mqxts\") pod \"kube-state-metrics-5dc8d7d89b-j6hsl\" (UID: \"0e786874-ce56-46f8-87f5-920dacacd9fc\") " pod="openshift-monitoring/kube-state-metrics-5dc8d7d89b-j6hsl"
Oct 30 18:47:08.393562 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.393517    5704 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 18:47:08.393562 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.393555    5704 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 18:47:08.393615 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.393564    5704 projected.go:196] Error preparing data for projected volume kube-api-access-vsptj for pod openshift-console/console-77b7f76bd4-6rfpv: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 18:47:08.393647 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.393598    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:08.89358883 +0000 UTC m=+1.839756733 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-vsptj" (UniqueName: "kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 18:47:08.396305 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.396288    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrrbp\" (UniqueName: \"kubernetes.io/projected/4cb022e6-43bd-4d71-9935-f0e91130735b-kube-api-access-vrrbp\") pod \"network-metrics-daemon-pgbfm\" (UID: \"4cb022e6-43bd-4d71-9935-f0e91130735b\") " pod="openshift-multus/network-metrics-daemon-pgbfm"
Oct 30 18:47:08.396490 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.396469    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp6fz\" (UniqueName: \"kubernetes.io/projected/3d07d694-4e2c-401a-aeab-0c9c4f530e87-kube-api-access-bp6fz\") pod \"dns-default-j88bw\" (UID: \"3d07d694-4e2c-401a-aeab-0c9c4f530e87\") " pod="openshift-dns/dns-default-j88bw"
Oct 30 18:47:08.396707 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.396693    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtds7\" (UniqueName: \"kubernetes.io/projected/42b2d2d6-78f0-457a-b182-09dd65dc3372-kube-api-access-jtds7\") pod \"openshift-state-metrics-7bd7489957-5v6xc\" (UID: \"42b2d2d6-78f0-457a-b182-09dd65dc3372\") " pod="openshift-monitoring/openshift-state-metrics-7bd7489957-5v6xc"
Oct 30 18:47:08.398489 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.398473    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2t26\" (UniqueName: \"kubernetes.io/projected/9ae72d3a-8b68-4d30-93d7-f03418559928-kube-api-access-z2t26\") pod \"thanos-querier-5d769f5588-7flhp\" (UID: \"9ae72d3a-8b68-4d30-93d7-f03418559928\") " pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:08.398636 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.398617    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8brjr\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-kube-api-access-8brjr\") pod \"alertmanager-main-0\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:47:08.398886 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.398873    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5229t\" (UniqueName: \"kubernetes.io/projected/ddbd05bb-e311-4a16-af9e-0ab5fa5103c0-kube-api-access-5229t\") pod \"network-check-target-42cjm\" (UID: \"ddbd05bb-e311-4a16-af9e-0ab5fa5103c0\") " pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:47:08.399594 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.399579    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6fjw\" (UniqueName: \"kubernetes.io/projected/738e26c2-0aac-4df0-961d-8c7404c8d6ab-kube-api-access-w6fjw\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:47:08.434566 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434544    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea684836-96a5-4d68-a414-c3db5de7564e-host-slash\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:47:08.434637 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434571    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-etc-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.434637 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434587    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9822bfc7-212f-4cf6-8d56-eb44c58c359b-host\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:47:08.434637 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434606    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-ovn\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.434637 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434629    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-systemd\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.434841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434650    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-etc-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.434841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434656    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9822bfc7-212f-4cf6-8d56-eb44c58c359b-host\") pod \"node-ca-664td\" (UID: \"9822bfc7-212f-4cf6-8d56-eb44c58c359b\") " pod="openshift-image-registry/node-ca-664td"
Oct 30 18:47:08.434841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434670    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-ovn\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.434841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434696    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-systemd\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.434841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434699    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea684836-96a5-4d68-a414-c3db5de7564e-host-slash\") pod \"iptables-alerter-v6t5q\" (UID: \"ea684836-96a5-4d68-a414-c3db5de7564e\") " pod="openshift-network-operator/iptables-alerter-v6t5q"
Oct 30 18:47:08.434841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434739    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cnibin\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.434841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434767    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/738e26c2-0aac-4df0-961d-8c7404c8d6ab-hosts-file\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:47:08.434841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434792    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-root\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.434841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434813    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-cnibin\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.434841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434831    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/738e26c2-0aac-4df0-961d-8c7404c8d6ab-hosts-file\") pod \"node-resolver-hx8pq\" (UID: \"738e26c2-0aac-4df0-961d-8c7404c8d6ab\") " pod="openshift-dns/node-resolver-hx8pq"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434854    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-root\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434884    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-socket-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434916    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-registration-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434940    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-etc-selinux\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434982    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysctl-d\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434994    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-registration-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435000    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-cni-netd\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435001    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-etc-selinux\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.434995    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-socket-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435019    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-cni-multus\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435023    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-cni-netd\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435058    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-cni-multus\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435073    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-conf-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435096    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysctl-d\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435101    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-node-log\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435137    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-conf-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435154    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-node-log\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435159    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-device-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435184    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-kubelet\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435198    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-sys\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435212    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-cni-bin\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435241    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-kubelet\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435248    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435255    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-sys\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435273    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-cni-bin\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435285    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-device-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.435274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435289    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435299    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-cni-bin\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435273    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-cni-bin\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435302    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-run\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435328    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-wtmp\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435334    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-run\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435345    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecbb798a-2024-4984-89c1-56ad55912e77-crio-socket\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435363    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-sys-fs\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435371    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/ecbb798a-2024-4984-89c1-56ad55912e77-crio-socket\") pod \"insights-runtime-extractor-6d2f8\" (UID: \"ecbb798a-2024-4984-89c1-56ad55912e77\") " pod="openshift-insights/insights-runtime-extractor-6d2f8"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435378    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435392    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-lib-modules\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435397    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-node-exporter-wtmp\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435406    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-sys-fs\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435407    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysconfig\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435426    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435428    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-hostroot\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435432    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysconfig\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435450    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-hostroot\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435457    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-multus-certs\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435475    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-run-netns\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435490    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-systemd\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435497    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-lib-modules\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435519    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-cni-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435533    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-run-netns\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435549    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-run-systemd\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435555    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-cni-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435569    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-slash\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435523    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-multus-certs\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435587    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-system-cni-dir\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435604    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435622    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-slash\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435627    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-log-socket\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435638    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-system-cni-dir\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435660    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-log-socket\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435661    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-modprobe-d\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435682    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysctl-conf\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435686    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3cbb0cdd-8ae2-4149-84d8-5df55b50e54a-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ph92c\" (UID: \"3cbb0cdd-8ae2-4149-84d8-5df55b50e54a\") " pod="openshift-multus/multus-additional-cni-plugins-ph92c"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435697    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-var-lib-kubelet\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435713    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-kubelet\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435737    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-sys\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435760    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435766    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-modprobe-d\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435771    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-var-lib-kubelet\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435789    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-var-lib-kubelet\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435798    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b67fee5e-0972-4cfc-987d-b0ae79e6589d-dbus\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435800    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-sysctl-conf\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435825    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ca586ba2-2d05-45f8-aab9-127b93ea1c60-sys\") pod \"node-exporter-qdr6m\" (UID: \"ca586ba2-2d05-45f8-aab9-127b93ea1c60\") " pod="openshift-monitoring/node-exporter-qdr6m"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435830    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-host\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435837    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b67fee5e-0972-4cfc-987d-b0ae79e6589d-dbus\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435864    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/cda0bc7c-777f-4965-b097-6b5f3b6bfc52-kubelet-dir\") pod \"aws-ebs-csi-driver-node-d5tjx\" (UID: \"cda0bc7c-777f-4965-b097-6b5f3b6bfc52\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-d5tjx"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435870    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-systemd-units\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435892    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-host\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435896    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-k8s-cni-cncf-io\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435923    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b67fee5e-0972-4cfc-987d-b0ae79e6589d-kubelet-config\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435940    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-systemd-units\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435951    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-k8s-cni-cncf-io\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435951    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-run-ovn-kubernetes\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435976    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-system-cni-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435977    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b67fee5e-0972-4cfc-987d-b0ae79e6589d-kubelet-config\") pod \"global-pull-secret-syncer-wpm6z\" (UID: \"b67fee5e-0972-4cfc-987d-b0ae79e6589d\") " pod="kube-system/global-pull-secret-syncer-wpm6z"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.435983    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-host-run-ovn-kubernetes\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.436001    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-cnibin\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.436012    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-system-cni-dir\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.436025    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-socket-dir-parent\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.436060    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-cnibin\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.436070    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-etc-kubernetes\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.436095    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-netns\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.436110    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-multus-socket-dir-parent\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.436117    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-var-lib-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.436151    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/652a5a58-ad99-49bb-b371-743c2816482d-var-lib-openvswitch\") pod \"ovnkube-node-45wjp\" (UID: \"652a5a58-ad99-49bb-b371-743c2816482d\") " pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.436153    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-host-run-netns\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.436159    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cc73f4b8-ee7e-4915-a883-e26fa87a89db-etc-kubernetes\") pod \"multus-vkvjm\" (UID: \"cc73f4b8-ee7e-4915-a883-e26fa87a89db\") " pod="openshift-multus/multus-vkvjm"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.436151    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-kubernetes\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.436315 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.436192    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/11550a43-fc10-43e9-835c-ae7c8b21be03-etc-kubernetes\") pod \"tuned-jddc7\" (UID: \"11550a43-fc10-43e9-835c-ae7c8b21be03\") " pod="openshift-cluster-node-tuning-operator/tuned-jddc7"
Oct 30 18:47:08.838817 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.838786    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.838968 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.838826    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.838968 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.838859    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.838968 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.838897    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.838968 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.838912    5704 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 18:47:08.838968 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.838964    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:09.838952134 +0000 UTC m=+2.785120019 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 18:47:08.839256 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.838978    5704 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 18:47:08.839256 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.838984    5704 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 18:47:08.839256 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.839014    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:09.8390044 +0000 UTC m=+2.785172281 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 18:47:08.839256 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.839024    5704 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 18:47:08.839256 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.839054    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:09.839021477 +0000 UTC m=+2.785189361 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 18:47:08.839256 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.839081    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.839256 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.839096    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:09.83908373 +0000 UTC m=+2.785251628 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 18:47:08.839256 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.839123    5704 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 18:47:08.839256 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.839140    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.839256 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.839176    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:09.839164673 +0000 UTC m=+2.785332554 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"service-ca" not registered
Oct 30 18:47:08.839256 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.839214    5704 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 18:47:08.839256 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.839257    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:09.839245499 +0000 UTC m=+2.785413394 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-config" not registered
Oct 30 18:47:08.940078 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.940054    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsptj\" (UniqueName: \"kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:08.940193 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.940170    5704 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 18:47:08.940193 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.940183    5704 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 18:47:08.940193 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.940192    5704 projected.go:196] Error preparing data for projected volume kube-api-access-vsptj for pod openshift-console/console-77b7f76bd4-6rfpv: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 18:47:08.940303 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:08.940228    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:09.940219068 +0000 UTC m=+2.886386969 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsptj" (UniqueName: "kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 18:47:08.980583 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:08.980565    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-85d7bcfb5c-6vmfw"
Oct 30 18:47:09.441243 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:09.441219    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:09.848943 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:09.848911    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:09.848943 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:09.848948    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:09.849222 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:09.848981    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:09.849222 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:09.849004    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:09.849222 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.849047    5704 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 18:47:09.849222 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.849075    5704 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 18:47:09.849222 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.849102    5704 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 18:47:09.849222 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.849113    5704 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 18:47:09.849222 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:09.849053    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:09.849222 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.849120    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:11.849103829 +0000 UTC m=+4.795271710 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 18:47:09.849222 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.849191    5704 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 18:47:09.849222 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.849202    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:11.849184073 +0000 UTC m=+4.795351958 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 18:47:09.849222 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.849222    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:11.849212259 +0000 UTC m=+4.795380141 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-config" not registered
Oct 30 18:47:09.849621 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.849246    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:11.849237261 +0000 UTC m=+4.795405147 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 18:47:09.849621 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.849266    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:11.849258738 +0000 UTC m=+4.795426649 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"service-ca" not registered
Oct 30 18:47:09.849621 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:09.849314    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:09.849621 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.849412    5704 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 18:47:09.849621 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.849450    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:11.849439164 +0000 UTC m=+4.795607049 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 18:47:09.950750 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:09.950716    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsptj\" (UniqueName: \"kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:09.950917 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.950891    5704 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 18:47:09.950917 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.950912    5704 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 18:47:09.951024 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.950926    5704 projected.go:196] Error preparing data for projected volume kube-api-access-vsptj for pod openshift-console/console-77b7f76bd4-6rfpv: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 18:47:09.951024 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:09.950987    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:11.950968098 +0000 UTC m=+4.897135993 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsptj" (UniqueName: "kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 18:47:10.488068 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:10.488009    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:10.669798 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:10.669768    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:10.954096 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:10.954071    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-5d769f5588-7flhp"
Oct 30 18:47:11.036590 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:11.036565    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-j88bw"
Oct 30 18:47:11.869024 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:11.868980    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.869077    5704 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:11.869096    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.869126    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:15.869113648 +0000 UTC m=+8.815281530 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-config" not registered
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:11.869149    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.869180    5704 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:11.869188    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:11.869207    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.869223    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:15.869209726 +0000 UTC m=+8.815377607 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.869254    5704 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.869282    5704 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:11.869284    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.869305    5704 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.869318    5704 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.869289    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:15.869278125 +0000 UTC m=+8.815446007 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.869361    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:15.869352553 +0000 UTC m=+8.815520436 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.869373    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:15.869367806 +0000 UTC m=+8.815535687 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"service-ca" not registered
Oct 30 18:47:11.869573 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.869390    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:15.869380823 +0000 UTC m=+8.815548708 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 18:47:11.885882 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:11.885853    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-77f4f76594-fnkq7"
Oct 30 18:47:11.969991 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:11.969963    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsptj\" (UniqueName: \"kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:11.970140 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.970124    5704 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 18:47:11.970200 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.970144    5704 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 18:47:11.970200 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.970153    5704 projected.go:196] Error preparing data for projected volume kube-api-access-vsptj for pod openshift-console/console-77b7f76bd4-6rfpv: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 18:47:11.970257 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:11.970200    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:15.970186381 +0000 UTC m=+8.916354279 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsptj" (UniqueName: "kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 18:47:13.199448 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:13.199420    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:13.199924 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:13.199610    5704 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness"
Oct 30 18:47:14.586168 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:14.586129    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:15.721383 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:15.721340    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:15.899231 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:15.899200    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:15.899231 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:15.899235    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:15.899462 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:15.899264    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:15.899462 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:15.899284    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:15.899462 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:15.899342    5704 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 18:47:15.899462 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:15.899351    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:15.899462 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:15.899373    5704 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 18:47:15.899462 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:15.899399    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:15.899462 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:15.899413    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:23.899394127 +0000 UTC m=+16.845562019 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 18:47:15.899462 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:15.899436    5704 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 18:47:15.899462 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:15.899345    5704 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 18:47:15.899462 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:15.899466    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:23.899449258 +0000 UTC m=+16.845617156 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 18:47:15.899836 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:15.899472    5704 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 18:47:15.899836 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:15.899484    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:23.89947568 +0000 UTC m=+16.845643565 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 18:47:15.899836 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:15.899484    5704 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 18:47:15.899836 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:15.899516    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:23.89949792 +0000 UTC m=+16.845665814 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-config" not registered
Oct 30 18:47:15.899836 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:15.899532    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:23.899526668 +0000 UTC m=+16.845694548 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 18:47:15.899836 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:15.899543    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:23.899537925 +0000 UTC m=+16.845705805 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"service-ca" not registered
Oct 30 18:47:16.000670 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:16.000647    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsptj\" (UniqueName: \"kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:16.000819 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:16.000804    5704 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 18:47:16.000894 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:16.000829    5704 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 18:47:16.000894 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:16.000842    5704 projected.go:196] Error preparing data for projected volume kube-api-access-vsptj for pod openshift-console/console-77b7f76bd4-6rfpv: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 18:47:16.001006 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:16.000893    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:24.000877371 +0000 UTC m=+16.947045299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsptj" (UniqueName: "kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 18:47:16.180908 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:16.180882    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-nv7ql"
Oct 30 18:47:16.444430 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:16.444361    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-42cjm"
Oct 30 18:47:23.965818 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:23.965740    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:23.965818 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:23.965780    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:23.965825    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:23.965865    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:23.965888    5704 configmap.go:193] Couldn't get configMap openshift-console/trusted-ca-bundle: object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:23.965910    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:23.965934    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:23.965983    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:39.965963025 +0000 UTC m=+32.912130925 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"trusted-ca-bundle" not registered
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:23.965995    5704 configmap.go:193] Couldn't get configMap openshift-console/console-config: object "openshift-console"/"console-config" not registered
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:23.965996    5704 configmap.go:193] Couldn't get configMap openshift-console/service-ca: object "openshift-console"/"service-ca" not registered
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:23.965894    5704 secret.go:189] Couldn't get secret openshift-console/console-serving-cert: object "openshift-console"/"console-serving-cert" not registered
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:23.965888    5704 configmap.go:193] Couldn't get configMap openshift-console/oauth-serving-cert: object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:23.966075    5704 secret.go:189] Couldn't get secret openshift-console/console-oauth-config: object "openshift-console"/"console-oauth-config" not registered
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:23.966069    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:39.966052225 +0000 UTC m=+32.912220123 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-config" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-config" not registered
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:23.966148    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:39.966131973 +0000 UTC m=+32.912299861 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-oauth-config" (UniqueName: "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-oauth-config" not registered
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:23.966170    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:39.96615938 +0000 UTC m=+32.912327265 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "console-serving-cert" (UniqueName: "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"console-serving-cert" not registered
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:23.966184    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:39.966176331 +0000 UTC m=+32.912344211 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "oauth-serving-cert" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"oauth-serving-cert" not registered
Oct 30 18:47:23.966322 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:23.966198    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:39.966190055 +0000 UTC m=+32.912357936 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca" (UniqueName: "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : object "openshift-console"/"service-ca" not registered
Oct 30 18:47:24.066693 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:24.066668    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vsptj\" (UniqueName: \"kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj\") pod \"console-77b7f76bd4-6rfpv\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") " pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:24.066843 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:24.066828    5704 projected.go:291] Couldn't get configMap openshift-console/kube-root-ca.crt: object "openshift-console"/"kube-root-ca.crt" not registered
Oct 30 18:47:24.066917 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:24.066848    5704 projected.go:291] Couldn't get configMap openshift-console/openshift-service-ca.crt: object "openshift-console"/"openshift-service-ca.crt" not registered
Oct 30 18:47:24.066917 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:24.066860    5704 projected.go:196] Error preparing data for projected volume kube-api-access-vsptj for pod openshift-console/console-77b7f76bd4-6rfpv: [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 18:47:24.066917 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:24.066909    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj podName:1f359854-4031-4cf7-9ef4-877e4a90d249 nodeName:}" failed. No retries permitted until 2026-10-30 18:47:40.066895372 +0000 UTC m=+33.013063274 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-vsptj" (UniqueName: "kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj") pod "console-77b7f76bd4-6rfpv" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249") : [object "openshift-console"/"kube-root-ca.crt" not registered, object "openshift-console"/"openshift-service-ca.crt" not registered]
Oct 30 18:47:24.424650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:24.424626    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-45wjp"
Oct 30 18:47:25.309441 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:25.309400    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:25.488071 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:25.488025    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:33.258559 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.258526    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-77b7f76bd4-6rfpv" podUID="1f359854-4031-4cf7-9ef4-877e4a90d249" containerName="console" containerID="cri-o://7522e19ea9d180138546b0d27e32ca0e83b22b254fe812a9c2916579fbe3b480" gracePeriod=15
Oct 30 18:47:33.382476 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:33.382442    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:47:33.496167 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.496146    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:33.651842 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.651778    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config\") pod \"1f359854-4031-4cf7-9ef4-877e4a90d249\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") "
Oct 30 18:47:33.651842 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.651831    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert\") pod \"1f359854-4031-4cf7-9ef4-877e4a90d249\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") "
Oct 30 18:47:33.652004 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.651892    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca\") pod \"1f359854-4031-4cf7-9ef4-877e4a90d249\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") "
Oct 30 18:47:33.652004 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.651922    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1f359854-4031-4cf7-9ef4-877e4a90d249-tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f359854-4031-4cf7-9ef4-877e4a90d249-tmp\") pod \"1f359854-4031-4cf7-9ef4-877e4a90d249\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") "
Oct 30 18:47:33.652004 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.651973    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert\") pod \"1f359854-4031-4cf7-9ef4-877e4a90d249\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") "
Oct 30 18:47:33.652155 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.652014    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config\") pod \"1f359854-4031-4cf7-9ef4-877e4a90d249\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") "
Oct 30 18:47:33.652155 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.652062    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle\") pod \"1f359854-4031-4cf7-9ef4-877e4a90d249\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") "
Oct 30 18:47:33.652155 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.652098    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj\" (UniqueName: \"kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj\") pod \"1f359854-4031-4cf7-9ef4-877e4a90d249\" (UID: \"1f359854-4031-4cf7-9ef4-877e4a90d249\") "
Oct 30 18:47:33.652479 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.652417    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1f359854-4031-4cf7-9ef4-877e4a90d249-tmp" pod "1f359854-4031-4cf7-9ef4-877e4a90d249" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:47:33.652479 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.652345    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca" pod "1f359854-4031-4cf7-9ef4-877e4a90d249" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:47:33.652479 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.652340    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert" pod "1f359854-4031-4cf7-9ef4-877e4a90d249" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:47:33.652683 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.652543    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle" pod "1f359854-4031-4cf7-9ef4-877e4a90d249" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:47:33.652683 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.652615    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config" pod "1f359854-4031-4cf7-9ef4-877e4a90d249" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:47:33.654293 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.654260    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config" pod "1f359854-4031-4cf7-9ef4-877e4a90d249" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:33.654384 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.654291    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj" pod "1f359854-4031-4cf7-9ef4-877e4a90d249" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249"). InnerVolumeSpecName "kube-api-access-vsptj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:47:33.654384 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.654319    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert" pod "1f359854-4031-4cf7-9ef4-877e4a90d249" (UID: "1f359854-4031-4cf7-9ef4-877e4a90d249"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:47:33.753493 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.753469    5704 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:33.753493 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.753488    5704 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f359854-4031-4cf7-9ef4-877e4a90d249-tmp\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:33.753620 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.753498    5704 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-serving-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:33.753620 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.753507    5704 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-console-config\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:33.753620 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.753516    5704 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-trusted-ca-bundle\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:33.753620 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.753526    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vsptj\" (UniqueName: \"kubernetes.io/projected/1f359854-4031-4cf7-9ef4-877e4a90d249-kube-api-access-vsptj\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:33.753620 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.753535    5704 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f359854-4031-4cf7-9ef4-877e4a90d249-console-oauth-config\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:33.753620 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:33.753545    5704 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f359854-4031-4cf7-9ef4-877e4a90d249-oauth-serving-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:47:34.330613 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:34.330580    5704 generic.go:356] "Generic (PLEG): container finished" podID="1f359854-4031-4cf7-9ef4-877e4a90d249" containerID="7522e19ea9d180138546b0d27e32ca0e83b22b254fe812a9c2916579fbe3b480" exitCode=0
Oct 30 18:47:34.331100 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:34.330634    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77b7f76bd4-6rfpv"
Oct 30 18:47:34.331100 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:34.330668    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b7f76bd4-6rfpv" event={"ID":"1f359854-4031-4cf7-9ef4-877e4a90d249","Type":"ContainerDied","Data":"7522e19ea9d180138546b0d27e32ca0e83b22b254fe812a9c2916579fbe3b480"}
Oct 30 18:47:34.331100 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:34.330723    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77b7f76bd4-6rfpv" event={"ID":"1f359854-4031-4cf7-9ef4-877e4a90d249","Type":"ContainerDied","Data":"7bc50e16469aee5af6b8dab32fa3c80a8f1923d1eb0f3763e875fc6c42bec2b8"}
Oct 30 18:47:34.331100 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:34.330744    5704 scope.go:122] "RemoveContainer" containerID="7522e19ea9d180138546b0d27e32ca0e83b22b254fe812a9c2916579fbe3b480"
Oct 30 18:47:34.339606 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:34.339586    5704 scope.go:122] "RemoveContainer" containerID="7522e19ea9d180138546b0d27e32ca0e83b22b254fe812a9c2916579fbe3b480"
Oct 30 18:47:34.339873 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:47:34.339851    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7522e19ea9d180138546b0d27e32ca0e83b22b254fe812a9c2916579fbe3b480\": container with ID starting with 7522e19ea9d180138546b0d27e32ca0e83b22b254fe812a9c2916579fbe3b480 not found: ID does not exist" containerID="7522e19ea9d180138546b0d27e32ca0e83b22b254fe812a9c2916579fbe3b480"
Oct 30 18:47:34.339967 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:34.339878    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7522e19ea9d180138546b0d27e32ca0e83b22b254fe812a9c2916579fbe3b480"} err="failed to get container status \"7522e19ea9d180138546b0d27e32ca0e83b22b254fe812a9c2916579fbe3b480\": rpc error: code = NotFound desc = could not find container \"7522e19ea9d180138546b0d27e32ca0e83b22b254fe812a9c2916579fbe3b480\": container with ID starting with 7522e19ea9d180138546b0d27e32ca0e83b22b254fe812a9c2916579fbe3b480 not found: ID does not exist"
Oct 30 18:47:34.483195 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:34.483171    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77b7f76bd4-6rfpv"]
Oct 30 18:47:34.529391 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:34.529363    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77b7f76bd4-6rfpv"]
Oct 30 18:47:35.176514 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:35.176481    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f359854-4031-4cf7-9ef4-877e4a90d249" path="/var/lib/kubelet/pods/1f359854-4031-4cf7-9ef4-877e4a90d249/volumes"
Oct 30 18:47:35.409104 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:35.409070    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:36.363200 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:36.363163    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:47.308464 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:47.308429    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:48.272887 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:48.272854    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:54.424404 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:54.424371    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:56.877601 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:56.877571    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:57.112515 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:57.112479    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:47:59.099812 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.099781    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:47:59.100219 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.100073    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="alertmanager" containerID="cri-o://029c727e72eaefa3b5c36c5839eb42561d83c1751195646b786093e959132bc8" gracePeriod=120
Oct 30 18:47:59.100219 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.100113    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="kube-rbac-proxy-metric" containerID="cri-o://c6f9238b7c08c2ba14f3ad3e2b6f1e5a9dab86fd86eda1100422a47632ad251c" gracePeriod=120
Oct 30 18:47:59.100219 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.100182    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="kube-rbac-proxy-web" containerID="cri-o://6a71f155d5d440b454c54c6f8a1bb50c234627d55602ed3bd9dc1277ac792466" gracePeriod=120
Oct 30 18:47:59.100759 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.100675    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="prom-label-proxy" containerID="cri-o://4c8e2d8371de748318149aeb5481965ee81592f199b46b0e6c6178107ef26634" gracePeriod=120
Oct 30 18:47:59.100759 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.100684    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="kube-rbac-proxy" containerID="cri-o://00bb712bc3823d540d10530c13b97af9a11ea6a844629276c63e4aa31ca2223e" gracePeriod=120
Oct 30 18:47:59.100893 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.100676    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="config-reloader" containerID="cri-o://7b4e8406dfb3b12b172d10289225b3d68a882a76e59a72e174d9f7af228969d5" gracePeriod=120
Oct 30 18:47:59.420666 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.420602    5704 generic.go:356] "Generic (PLEG): container finished" podID="2717f98e-b200-43c5-a089-73fb6a522933" containerID="4c8e2d8371de748318149aeb5481965ee81592f199b46b0e6c6178107ef26634" exitCode=0
Oct 30 18:47:59.420666 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.420626    5704 generic.go:356] "Generic (PLEG): container finished" podID="2717f98e-b200-43c5-a089-73fb6a522933" containerID="00bb712bc3823d540d10530c13b97af9a11ea6a844629276c63e4aa31ca2223e" exitCode=0
Oct 30 18:47:59.420666 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.420632    5704 generic.go:356] "Generic (PLEG): container finished" podID="2717f98e-b200-43c5-a089-73fb6a522933" containerID="6a71f155d5d440b454c54c6f8a1bb50c234627d55602ed3bd9dc1277ac792466" exitCode=0
Oct 30 18:47:59.420666 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.420638    5704 generic.go:356] "Generic (PLEG): container finished" podID="2717f98e-b200-43c5-a089-73fb6a522933" containerID="7b4e8406dfb3b12b172d10289225b3d68a882a76e59a72e174d9f7af228969d5" exitCode=0
Oct 30 18:47:59.420666 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.420644    5704 generic.go:356] "Generic (PLEG): container finished" podID="2717f98e-b200-43c5-a089-73fb6a522933" containerID="029c727e72eaefa3b5c36c5839eb42561d83c1751195646b786093e959132bc8" exitCode=0
Oct 30 18:47:59.420915 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.420670    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerDied","Data":"4c8e2d8371de748318149aeb5481965ee81592f199b46b0e6c6178107ef26634"}
Oct 30 18:47:59.420915 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.420728    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerDied","Data":"00bb712bc3823d540d10530c13b97af9a11ea6a844629276c63e4aa31ca2223e"}
Oct 30 18:47:59.420915 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.420754    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerDied","Data":"6a71f155d5d440b454c54c6f8a1bb50c234627d55602ed3bd9dc1277ac792466"}
Oct 30 18:47:59.420915 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.420779    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerDied","Data":"7b4e8406dfb3b12b172d10289225b3d68a882a76e59a72e174d9f7af228969d5"}
Oct 30 18:47:59.420915 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:47:59.420801    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerDied","Data":"029c727e72eaefa3b5c36c5839eb42561d83c1751195646b786093e959132bc8"}
Oct 30 18:48:00.340088 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.340064    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:00.426903 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.426823    5704 generic.go:356] "Generic (PLEG): container finished" podID="2717f98e-b200-43c5-a089-73fb6a522933" containerID="c6f9238b7c08c2ba14f3ad3e2b6f1e5a9dab86fd86eda1100422a47632ad251c" exitCode=0
Oct 30 18:48:00.426903 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.426880    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerDied","Data":"c6f9238b7c08c2ba14f3ad3e2b6f1e5a9dab86fd86eda1100422a47632ad251c"}
Oct 30 18:48:00.427076 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.426919    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:00.427076 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.426949    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2717f98e-b200-43c5-a089-73fb6a522933","Type":"ContainerDied","Data":"a5dcaf13f72dae8ace0899f1b37e8f9af530604a73d38e414d2bdf39ba069334"}
Oct 30 18:48:00.427076 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.426998    5704 scope.go:122] "RemoveContainer" containerID="4c8e2d8371de748318149aeb5481965ee81592f199b46b0e6c6178107ef26634"
Oct 30 18:48:00.432136 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432114    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-metrics-client-ca\") pod \"2717f98e-b200-43c5-a089-73fb6a522933\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") "
Oct 30 18:48:00.432214 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432151    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-cluster-tls-config\") pod \"2717f98e-b200-43c5-a089-73fb6a522933\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") "
Oct 30 18:48:00.432214 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432179    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-metric\") pod \"2717f98e-b200-43c5-a089-73fb6a522933\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") "
Oct 30 18:48:00.432214 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432203    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-kube-api-access-8brjr\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-kube-api-access-8brjr\") pod \"2717f98e-b200-43c5-a089-73fb6a522933\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") "
Oct 30 18:48:00.432385 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432241    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-main-tls\") pod \"2717f98e-b200-43c5-a089-73fb6a522933\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") "
Oct 30 18:48:00.432385 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432271    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-web\") pod \"2717f98e-b200-43c5-a089-73fb6a522933\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") "
Oct 30 18:48:00.432385 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432309    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-main-db\") pod \"2717f98e-b200-43c5-a089-73fb6a522933\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") "
Oct 30 18:48:00.432385 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432350    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-config-out\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-config-out\") pod \"2717f98e-b200-43c5-a089-73fb6a522933\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") "
Oct 30 18:48:00.432385 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432382    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-config-volume\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-config-volume\") pod \"2717f98e-b200-43c5-a089-73fb6a522933\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") "
Oct 30 18:48:00.432652 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432420    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-trusted-ca-bundle\") pod \"2717f98e-b200-43c5-a089-73fb6a522933\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") "
Oct 30 18:48:00.432652 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432446    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-web-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-web-config\") pod \"2717f98e-b200-43c5-a089-73fb6a522933\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") "
Oct 30 18:48:00.432652 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432477    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-tls-assets\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-tls-assets\") pod \"2717f98e-b200-43c5-a089-73fb6a522933\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") "
Oct 30 18:48:00.432652 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432528    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-metrics-client-ca" pod "2717f98e-b200-43c5-a089-73fb6a522933" (UID: "2717f98e-b200-43c5-a089-73fb6a522933"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:00.432652 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432554    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy\") pod \"2717f98e-b200-43c5-a089-73fb6a522933\" (UID: \"2717f98e-b200-43c5-a089-73fb6a522933\") "
Oct 30 18:48:00.432959 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.432777    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-metrics-client-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:00.434245 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.433277    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-trusted-ca-bundle" pod "2717f98e-b200-43c5-a089-73fb6a522933" (UID: "2717f98e-b200-43c5-a089-73fb6a522933"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:48:00.434245 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.433659    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-main-db" pod "2717f98e-b200-43c5-a089-73fb6a522933" (UID: "2717f98e-b200-43c5-a089-73fb6a522933"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:00.435545 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.435516    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy" pod "2717f98e-b200-43c5-a089-73fb6a522933" (UID: "2717f98e-b200-43c5-a089-73fb6a522933"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:00.436085 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.436053    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-tls-assets" pod "2717f98e-b200-43c5-a089-73fb6a522933" (UID: "2717f98e-b200-43c5-a089-73fb6a522933"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:00.436174 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.436126    5704 scope.go:122] "RemoveContainer" containerID="c6f9238b7c08c2ba14f3ad3e2b6f1e5a9dab86fd86eda1100422a47632ad251c"
Oct 30 18:48:00.437757 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.437734    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-main-tls" pod "2717f98e-b200-43c5-a089-73fb6a522933" (UID: "2717f98e-b200-43c5-a089-73fb6a522933"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:00.437888 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.437762    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-config-out" pod "2717f98e-b200-43c5-a089-73fb6a522933" (UID: "2717f98e-b200-43c5-a089-73fb6a522933"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:48:00.437888 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.437810    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-config-volume" pod "2717f98e-b200-43c5-a089-73fb6a522933" (UID: "2717f98e-b200-43c5-a089-73fb6a522933"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:00.441409 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.441350    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-kube-api-access-8brjr" pod "2717f98e-b200-43c5-a089-73fb6a522933" (UID: "2717f98e-b200-43c5-a089-73fb6a522933"). InnerVolumeSpecName "kube-api-access-8brjr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:48:00.441409 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.441374    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-metric" pod "2717f98e-b200-43c5-a089-73fb6a522933" (UID: "2717f98e-b200-43c5-a089-73fb6a522933"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:00.442953 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.442915    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-web" pod "2717f98e-b200-43c5-a089-73fb6a522933" (UID: "2717f98e-b200-43c5-a089-73fb6a522933"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:00.443759 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.443740    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-cluster-tls-config" pod "2717f98e-b200-43c5-a089-73fb6a522933" (UID: "2717f98e-b200-43c5-a089-73fb6a522933"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:00.446512 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.446490    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-web-config" pod "2717f98e-b200-43c5-a089-73fb6a522933" (UID: "2717f98e-b200-43c5-a089-73fb6a522933"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:48:00.455344 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.455326    5704 scope.go:122] "RemoveContainer" containerID="00bb712bc3823d540d10530c13b97af9a11ea6a844629276c63e4aa31ca2223e"
Oct 30 18:48:00.461776 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.461763    5704 scope.go:122] "RemoveContainer" containerID="6a71f155d5d440b454c54c6f8a1bb50c234627d55602ed3bd9dc1277ac792466"
Oct 30 18:48:00.467781 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.467762    5704 scope.go:122] "RemoveContainer" containerID="7b4e8406dfb3b12b172d10289225b3d68a882a76e59a72e174d9f7af228969d5"
Oct 30 18:48:00.474084 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.474041    5704 scope.go:122] "RemoveContainer" containerID="029c727e72eaefa3b5c36c5839eb42561d83c1751195646b786093e959132bc8"
Oct 30 18:48:00.480399 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.480386    5704 scope.go:122] "RemoveContainer" containerID="070fcf583e97a5546d1b72cba3246e572a54913e0c473fbbb39dd8e861eaa71f"
Oct 30 18:48:00.486639 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.486623    5704 scope.go:122] "RemoveContainer" containerID="4c8e2d8371de748318149aeb5481965ee81592f199b46b0e6c6178107ef26634"
Oct 30 18:48:00.486871 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:48:00.486856    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c8e2d8371de748318149aeb5481965ee81592f199b46b0e6c6178107ef26634\": container with ID starting with 4c8e2d8371de748318149aeb5481965ee81592f199b46b0e6c6178107ef26634 not found: ID does not exist" containerID="4c8e2d8371de748318149aeb5481965ee81592f199b46b0e6c6178107ef26634"
Oct 30 18:48:00.486920 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.486880    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c8e2d8371de748318149aeb5481965ee81592f199b46b0e6c6178107ef26634"} err="failed to get container status \"4c8e2d8371de748318149aeb5481965ee81592f199b46b0e6c6178107ef26634\": rpc error: code = NotFound desc = could not find container \"4c8e2d8371de748318149aeb5481965ee81592f199b46b0e6c6178107ef26634\": container with ID starting with 4c8e2d8371de748318149aeb5481965ee81592f199b46b0e6c6178107ef26634 not found: ID does not exist"
Oct 30 18:48:00.486920 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.486897    5704 scope.go:122] "RemoveContainer" containerID="c6f9238b7c08c2ba14f3ad3e2b6f1e5a9dab86fd86eda1100422a47632ad251c"
Oct 30 18:48:00.487195 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:48:00.487180    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c6f9238b7c08c2ba14f3ad3e2b6f1e5a9dab86fd86eda1100422a47632ad251c\": container with ID starting with c6f9238b7c08c2ba14f3ad3e2b6f1e5a9dab86fd86eda1100422a47632ad251c not found: ID does not exist" containerID="c6f9238b7c08c2ba14f3ad3e2b6f1e5a9dab86fd86eda1100422a47632ad251c"
Oct 30 18:48:00.487240 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.487201    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c6f9238b7c08c2ba14f3ad3e2b6f1e5a9dab86fd86eda1100422a47632ad251c"} err="failed to get container status \"c6f9238b7c08c2ba14f3ad3e2b6f1e5a9dab86fd86eda1100422a47632ad251c\": rpc error: code = NotFound desc = could not find container \"c6f9238b7c08c2ba14f3ad3e2b6f1e5a9dab86fd86eda1100422a47632ad251c\": container with ID starting with c6f9238b7c08c2ba14f3ad3e2b6f1e5a9dab86fd86eda1100422a47632ad251c not found: ID does not exist"
Oct 30 18:48:00.487240 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.487214    5704 scope.go:122] "RemoveContainer" containerID="00bb712bc3823d540d10530c13b97af9a11ea6a844629276c63e4aa31ca2223e"
Oct 30 18:48:00.487409 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:48:00.487395    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00bb712bc3823d540d10530c13b97af9a11ea6a844629276c63e4aa31ca2223e\": container with ID starting with 00bb712bc3823d540d10530c13b97af9a11ea6a844629276c63e4aa31ca2223e not found: ID does not exist" containerID="00bb712bc3823d540d10530c13b97af9a11ea6a844629276c63e4aa31ca2223e"
Oct 30 18:48:00.487451 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.487413    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00bb712bc3823d540d10530c13b97af9a11ea6a844629276c63e4aa31ca2223e"} err="failed to get container status \"00bb712bc3823d540d10530c13b97af9a11ea6a844629276c63e4aa31ca2223e\": rpc error: code = NotFound desc = could not find container \"00bb712bc3823d540d10530c13b97af9a11ea6a844629276c63e4aa31ca2223e\": container with ID starting with 00bb712bc3823d540d10530c13b97af9a11ea6a844629276c63e4aa31ca2223e not found: ID does not exist"
Oct 30 18:48:00.487451 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.487424    5704 scope.go:122] "RemoveContainer" containerID="6a71f155d5d440b454c54c6f8a1bb50c234627d55602ed3bd9dc1277ac792466"
Oct 30 18:48:00.487618 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:48:00.487603    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a71f155d5d440b454c54c6f8a1bb50c234627d55602ed3bd9dc1277ac792466\": container with ID starting with 6a71f155d5d440b454c54c6f8a1bb50c234627d55602ed3bd9dc1277ac792466 not found: ID does not exist" containerID="6a71f155d5d440b454c54c6f8a1bb50c234627d55602ed3bd9dc1277ac792466"
Oct 30 18:48:00.487656 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.487621    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a71f155d5d440b454c54c6f8a1bb50c234627d55602ed3bd9dc1277ac792466"} err="failed to get container status \"6a71f155d5d440b454c54c6f8a1bb50c234627d55602ed3bd9dc1277ac792466\": rpc error: code = NotFound desc = could not find container \"6a71f155d5d440b454c54c6f8a1bb50c234627d55602ed3bd9dc1277ac792466\": container with ID starting with 6a71f155d5d440b454c54c6f8a1bb50c234627d55602ed3bd9dc1277ac792466 not found: ID does not exist"
Oct 30 18:48:00.487656 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.487632    5704 scope.go:122] "RemoveContainer" containerID="7b4e8406dfb3b12b172d10289225b3d68a882a76e59a72e174d9f7af228969d5"
Oct 30 18:48:00.487822 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:48:00.487808    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b4e8406dfb3b12b172d10289225b3d68a882a76e59a72e174d9f7af228969d5\": container with ID starting with 7b4e8406dfb3b12b172d10289225b3d68a882a76e59a72e174d9f7af228969d5 not found: ID does not exist" containerID="7b4e8406dfb3b12b172d10289225b3d68a882a76e59a72e174d9f7af228969d5"
Oct 30 18:48:00.487862 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.487825    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b4e8406dfb3b12b172d10289225b3d68a882a76e59a72e174d9f7af228969d5"} err="failed to get container status \"7b4e8406dfb3b12b172d10289225b3d68a882a76e59a72e174d9f7af228969d5\": rpc error: code = NotFound desc = could not find container \"7b4e8406dfb3b12b172d10289225b3d68a882a76e59a72e174d9f7af228969d5\": container with ID starting with 7b4e8406dfb3b12b172d10289225b3d68a882a76e59a72e174d9f7af228969d5 not found: ID does not exist"
Oct 30 18:48:00.487862 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.487836    5704 scope.go:122] "RemoveContainer" containerID="029c727e72eaefa3b5c36c5839eb42561d83c1751195646b786093e959132bc8"
Oct 30 18:48:00.488016 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:48:00.488002    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"029c727e72eaefa3b5c36c5839eb42561d83c1751195646b786093e959132bc8\": container with ID starting with 029c727e72eaefa3b5c36c5839eb42561d83c1751195646b786093e959132bc8 not found: ID does not exist" containerID="029c727e72eaefa3b5c36c5839eb42561d83c1751195646b786093e959132bc8"
Oct 30 18:48:00.488066 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.488018    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"029c727e72eaefa3b5c36c5839eb42561d83c1751195646b786093e959132bc8"} err="failed to get container status \"029c727e72eaefa3b5c36c5839eb42561d83c1751195646b786093e959132bc8\": rpc error: code = NotFound desc = could not find container \"029c727e72eaefa3b5c36c5839eb42561d83c1751195646b786093e959132bc8\": container with ID starting with 029c727e72eaefa3b5c36c5839eb42561d83c1751195646b786093e959132bc8 not found: ID does not exist"
Oct 30 18:48:00.488128 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.488117    5704 scope.go:122] "RemoveContainer" containerID="070fcf583e97a5546d1b72cba3246e572a54913e0c473fbbb39dd8e861eaa71f"
Oct 30 18:48:00.488324 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:48:00.488306    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"070fcf583e97a5546d1b72cba3246e572a54913e0c473fbbb39dd8e861eaa71f\": container with ID starting with 070fcf583e97a5546d1b72cba3246e572a54913e0c473fbbb39dd8e861eaa71f not found: ID does not exist" containerID="070fcf583e97a5546d1b72cba3246e572a54913e0c473fbbb39dd8e861eaa71f"
Oct 30 18:48:00.488360 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.488331    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"070fcf583e97a5546d1b72cba3246e572a54913e0c473fbbb39dd8e861eaa71f"} err="failed to get container status \"070fcf583e97a5546d1b72cba3246e572a54913e0c473fbbb39dd8e861eaa71f\": rpc error: code = NotFound desc = could not find container \"070fcf583e97a5546d1b72cba3246e572a54913e0c473fbbb39dd8e861eaa71f\": container with ID starting with 070fcf583e97a5546d1b72cba3246e572a54913e0c473fbbb39dd8e861eaa71f not found: ID does not exist"
Oct 30 18:48:00.533380 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.533345    5704 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:00.533467 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.533393    5704 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-web-config\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:00.533467 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.533405    5704 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-tls-assets\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:00.533467 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.533414    5704 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:00.533467 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.533424    5704 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-cluster-tls-config\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:00.533467 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.533433    5704 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:00.533467 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.533444    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8brjr\" (UniqueName: \"kubernetes.io/projected/2717f98e-b200-43c5-a089-73fb6a522933-kube-api-access-8brjr\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:00.533467 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.533452    5704 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-main-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:00.533467 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.533462    5704 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:00.533740 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.533471    5704 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-alertmanager-main-db\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:00.533740 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.533481    5704 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2717f98e-b200-43c5-a089-73fb6a522933-config-out\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:00.533740 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.533489    5704 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2717f98e-b200-43c5-a089-73fb6a522933-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:48:00.839792 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.839765    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:48:00.844401 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.844381    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:48:00.883434 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883412    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:48:00.883657 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883646    5704 cpu_manager.go:414] "containerMap: removing container" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="kube-rbac-proxy"
Oct 30 18:48:00.883698 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883658    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="kube-rbac-proxy"
Oct 30 18:48:00.883698 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883669    5704 cpu_manager.go:414] "containerMap: removing container" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="alertmanager"
Oct 30 18:48:00.883698 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883676    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="alertmanager"
Oct 30 18:48:00.883698 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883687    5704 cpu_manager.go:414] "containerMap: removing container" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="prom-label-proxy"
Oct 30 18:48:00.883698 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883692    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="prom-label-proxy"
Oct 30 18:48:00.883698 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883699    5704 cpu_manager.go:414] "containerMap: removing container" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="kube-rbac-proxy-web"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883704    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="kube-rbac-proxy-web"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883715    5704 cpu_manager.go:414] "containerMap: removing container" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="kube-rbac-proxy-metric"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883719    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="kube-rbac-proxy-metric"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883727    5704 cpu_manager.go:414] "containerMap: removing container" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="init-config-reloader"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883731    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="init-config-reloader"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883738    5704 cpu_manager.go:414] "containerMap: removing container" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="config-reloader"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883742    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="config-reloader"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883751    5704 cpu_manager.go:414] "containerMap: removing container" podUID="1f359854-4031-4cf7-9ef4-877e4a90d249" containerName="console"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883755    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f359854-4031-4cf7-9ef4-877e4a90d249" containerName="console"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883791    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="kube-rbac-proxy-metric"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883798    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="init-config-reloader"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883804    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="config-reloader"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883810    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f359854-4031-4cf7-9ef4-877e4a90d249" containerName="console"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883817    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="kube-rbac-proxy"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883822    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="alertmanager"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883829    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="prom-label-proxy"
Oct 30 18:48:00.883869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.883834    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="2717f98e-b200-43c5-a089-73fb6a522933" containerName="kube-rbac-proxy-web"
Oct 30 18:48:00.892407 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.888579    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:00.937617 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.937581    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Oct 30 18:48:00.937617 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.937595    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Oct 30 18:48:00.937853 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.937733    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Oct 30 18:48:00.938107 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.938086    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Oct 30 18:48:00.938107 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.938107    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Oct 30 18:48:00.938251 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.938119    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Oct 30 18:48:00.938413 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.938399    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Oct 30 18:48:00.938641 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.938628    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Oct 30 18:48:00.938721 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.938708    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-mgrlf\""
Oct 30 18:48:00.938772 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.938756    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Oct 30 18:48:00.984142 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:00.984121    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:48:01.037104 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.037079    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.037229 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.037119    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fffcbe9-25c2-4796-812a-f7ac6278ceab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.037229 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.037144    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbhvk\" (UniqueName: \"kubernetes.io/projected/2fffcbe9-25c2-4796-812a-f7ac6278ceab-kube-api-access-xbhvk\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.037229 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.037207    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2fffcbe9-25c2-4796-812a-f7ac6278ceab-config-out\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.037346 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.037261    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.037346 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.037296    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.037346 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.037312    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fffcbe9-25c2-4796-812a-f7ac6278ceab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.037443 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.037346    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-web-config\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.037443 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.037368    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.037443 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.037382    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2fffcbe9-25c2-4796-812a-f7ac6278ceab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.037531 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.037455    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.037531 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.037494    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2fffcbe9-25c2-4796-812a-f7ac6278ceab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.037602 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.037554    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-config-volume\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.138747 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.138723    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.138837 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.138751    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.138837 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.138767    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fffcbe9-25c2-4796-812a-f7ac6278ceab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.138837 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.138787    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-web-config\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.138837 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.138809    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.138996 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.138918    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2fffcbe9-25c2-4796-812a-f7ac6278ceab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.138996 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.138955    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.138996 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.138981    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2fffcbe9-25c2-4796-812a-f7ac6278ceab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.139189 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.139017    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-config-volume\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.139189 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.139060    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.139189 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.139093    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fffcbe9-25c2-4796-812a-f7ac6278ceab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.139189 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.139114    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbhvk\" (UniqueName: \"kubernetes.io/projected/2fffcbe9-25c2-4796-812a-f7ac6278ceab-kube-api-access-xbhvk\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.139189 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.139138    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2fffcbe9-25c2-4796-812a-f7ac6278ceab-config-out\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.139422 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.139182    5704 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Oct 30 18:48:01.139906 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.139598    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2fffcbe9-25c2-4796-812a-f7ac6278ceab-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.139906 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.139607    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/2fffcbe9-25c2-4796-812a-f7ac6278ceab-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.140492 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.140468    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2fffcbe9-25c2-4796-812a-f7ac6278ceab-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.142252 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.142224    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.142364 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.142259    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.142364 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.142289    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.142508 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.142490    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/2fffcbe9-25c2-4796-812a-f7ac6278ceab-config-out\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.142719 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.142697    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/2fffcbe9-25c2-4796-812a-f7ac6278ceab-tls-assets\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.142792 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.142750    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-config-volume\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.142858 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.142841    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-web-config\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.143638 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.143617    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.143909 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.143881    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/2fffcbe9-25c2-4796-812a-f7ac6278ceab-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.176447 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.176424    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2717f98e-b200-43c5-a089-73fb6a522933" path="/var/lib/kubelet/pods/2717f98e-b200-43c5-a089-73fb6a522933/volumes"
Oct 30 18:48:01.197704 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.197682    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbhvk\" (UniqueName: \"kubernetes.io/projected/2fffcbe9-25c2-4796-812a-f7ac6278ceab-kube-api-access-xbhvk\") pod \"alertmanager-main-0\" (UID: \"2fffcbe9-25c2-4796-812a-f7ac6278ceab\") " pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.210371 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.210344    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Oct 30 18:48:01.643266 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:01.643233    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Oct 30 18:48:01.650594 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:48:01.650567    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fffcbe9_25c2_4796_812a_f7ac6278ceab.slice/crio-f5449926dff63a0fdcf485a9171c3d35534a98546012de24f1bc753c5e59943f WatchSource:0}: Error finding container f5449926dff63a0fdcf485a9171c3d35534a98546012de24f1bc753c5e59943f: Status 404 returned error can't find the container with id f5449926dff63a0fdcf485a9171c3d35534a98546012de24f1bc753c5e59943f
Oct 30 18:48:02.436438 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.436411    5704 generic.go:356] "Generic (PLEG): container finished" podID="2fffcbe9-25c2-4796-812a-f7ac6278ceab" containerID="92c00af977c33b9a6b4dc34719e46ace9a9f6b0015a89bc0655ed5e37c91c8b2" exitCode=0
Oct 30 18:48:02.436606 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.436469    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fffcbe9-25c2-4796-812a-f7ac6278ceab","Type":"ContainerDied","Data":"92c00af977c33b9a6b4dc34719e46ace9a9f6b0015a89bc0655ed5e37c91c8b2"}
Oct 30 18:48:02.436606 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.436518    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fffcbe9-25c2-4796-812a-f7ac6278ceab","Type":"ContainerStarted","Data":"f5449926dff63a0fdcf485a9171c3d35534a98546012de24f1bc753c5e59943f"}
Oct 30 18:48:02.740071 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.739766    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"]
Oct 30 18:48:02.742363 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.742345    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.791195 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.791173    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Oct 30 18:48:02.791688 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.791671    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Oct 30 18:48:02.791756 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.791670    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Oct 30 18:48:02.791756 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.791719    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Oct 30 18:48:02.791841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.791813    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Oct 30 18:48:02.791919 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.791906    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-hg6wr\""
Oct 30 18:48:02.838356 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.838332    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-28r105lbnph2c\""
Oct 30 18:48:02.850709 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.850685    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/287c6859-11cf-4a66-a3f8-b866c5477bb1-secret-telemeter-client\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.850811 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.850719    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tfc7\" (UniqueName: \"kubernetes.io/projected/287c6859-11cf-4a66-a3f8-b866c5477bb1-kube-api-access-6tfc7\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.850811 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.850751    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/287c6859-11cf-4a66-a3f8-b866c5477bb1-serving-certs-ca-bundle\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.850909 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.850827    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/287c6859-11cf-4a66-a3f8-b866c5477bb1-telemeter-client-tls\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.850992 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.850974    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/287c6859-11cf-4a66-a3f8-b866c5477bb1-metrics-client-ca\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.851029 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.851015    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/287c6859-11cf-4a66-a3f8-b866c5477bb1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.851161 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.851144    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/287c6859-11cf-4a66-a3f8-b866c5477bb1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.851199 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.851184    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/287c6859-11cf-4a66-a3f8-b866c5477bb1-federate-client-tls\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.851530 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.851514    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"]
Oct 30 18:48:02.952276 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.952220    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/287c6859-11cf-4a66-a3f8-b866c5477bb1-secret-telemeter-client\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.952276 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.952244    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tfc7\" (UniqueName: \"kubernetes.io/projected/287c6859-11cf-4a66-a3f8-b866c5477bb1-kube-api-access-6tfc7\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.952276 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.952264    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/287c6859-11cf-4a66-a3f8-b866c5477bb1-serving-certs-ca-bundle\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.952474 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.952280    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/287c6859-11cf-4a66-a3f8-b866c5477bb1-telemeter-client-tls\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.952474 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.952320    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/287c6859-11cf-4a66-a3f8-b866c5477bb1-metrics-client-ca\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.952474 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.952349    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/287c6859-11cf-4a66-a3f8-b866c5477bb1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.952474 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.952405    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/287c6859-11cf-4a66-a3f8-b866c5477bb1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.952474 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.952436    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/287c6859-11cf-4a66-a3f8-b866c5477bb1-federate-client-tls\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.953153 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.953130    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/287c6859-11cf-4a66-a3f8-b866c5477bb1-serving-certs-ca-bundle\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.953257 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.953188    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/287c6859-11cf-4a66-a3f8-b866c5477bb1-metrics-client-ca\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.953360 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.953334    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/287c6859-11cf-4a66-a3f8-b866c5477bb1-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.954736 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.954714    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/287c6859-11cf-4a66-a3f8-b866c5477bb1-secret-telemeter-client\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.954893 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.954875    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/287c6859-11cf-4a66-a3f8-b866c5477bb1-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.955230 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.955213    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/287c6859-11cf-4a66-a3f8-b866c5477bb1-federate-client-tls\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:02.955298 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:02.955283    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/287c6859-11cf-4a66-a3f8-b866c5477bb1-telemeter-client-tls\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:03.009671 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:03.009644    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tfc7\" (UniqueName: \"kubernetes.io/projected/287c6859-11cf-4a66-a3f8-b866c5477bb1-kube-api-access-6tfc7\") pod \"telemeter-client-76b58b9cbb-gctk8\" (UID: \"287c6859-11cf-4a66-a3f8-b866c5477bb1\") " pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:03.055401 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:03.055381    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"
Oct 30 18:48:03.380903 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:48:03.380869    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:48:03.442928 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:03.442896    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fffcbe9-25c2-4796-812a-f7ac6278ceab","Type":"ContainerStarted","Data":"912c9e7a4aa58236a16c0ac413a71182a11208e67c8990db1a547ed33bea2516"}
Oct 30 18:48:03.443052 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:03.442959    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fffcbe9-25c2-4796-812a-f7ac6278ceab","Type":"ContainerStarted","Data":"2775d94055727b8a78ff09bb96e02df61c0d6400ba8d2bae8fca4e7ba86c7e8a"}
Oct 30 18:48:03.443052 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:03.442985    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fffcbe9-25c2-4796-812a-f7ac6278ceab","Type":"ContainerStarted","Data":"8546eeb1d1d92303c6f1b715a686961d05072ebce24e5af24a07582e138c2eb9"}
Oct 30 18:48:03.443052 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:03.443012    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fffcbe9-25c2-4796-812a-f7ac6278ceab","Type":"ContainerStarted","Data":"41723564a4c4504a025721ee9cf9dded934fa46a40090b5036c1b081d6e37961"}
Oct 30 18:48:03.443052 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:03.443049    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fffcbe9-25c2-4796-812a-f7ac6278ceab","Type":"ContainerStarted","Data":"7bf65d680c4b5578a8615beec7ec21542c8c68fe999230dfb4f55472a3f79394"}
Oct 30 18:48:03.443186 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:03.443074    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"2fffcbe9-25c2-4796-812a-f7ac6278ceab","Type":"ContainerStarted","Data":"8051d279e9c5a65b26953f4f386a56d8c042303dc8d5440d4f226ea4adc6b9e7"}
Oct 30 18:48:03.479688 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:03.479666    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76b58b9cbb-gctk8"]
Oct 30 18:48:03.487296 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:48:03.487270    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod287c6859_11cf_4a66_a3f8_b866c5477bb1.slice/crio-f4bcf7a7f64ccf35c747cf05ec6e7748987ffc991021741ce2de2c96f4e139ea WatchSource:0}: Error finding container f4bcf7a7f64ccf35c747cf05ec6e7748987ffc991021741ce2de2c96f4e139ea: Status 404 returned error can't find the container with id f4bcf7a7f64ccf35c747cf05ec6e7748987ffc991021741ce2de2c96f4e139ea
Oct 30 18:48:03.489103 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:03.489084    5704 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:48:03.569750 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:03.569711    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.559556049 podStartE2EDuration="3.569697773s" totalImagesPullingTime="0s" totalInitContainerRuntime="10.141724ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:03.568762457 +0000 UTC m=+56.514930359" watchObservedRunningTime="2026-10-30 18:48:03.569697773 +0000 UTC m=+56.515865672"
Oct 30 18:48:04.447381 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:04.447347    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8" event={"ID":"287c6859-11cf-4a66-a3f8-b866c5477bb1","Type":"ContainerStarted","Data":"f4bcf7a7f64ccf35c747cf05ec6e7748987ffc991021741ce2de2c96f4e139ea"}
Oct 30 18:48:06.455730 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:06.455660    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8" event={"ID":"287c6859-11cf-4a66-a3f8-b866c5477bb1","Type":"ContainerStarted","Data":"822ae039a6c10e75ad16216d88dae9d4c4361acdb6c3221413c22b1d65d437ee"}
Oct 30 18:48:06.455730 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:06.455711    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8" event={"ID":"287c6859-11cf-4a66-a3f8-b866c5477bb1","Type":"ContainerStarted","Data":"1329a7df0a4e89650e5e9c53bc21f227f4ba6dc291ddb19d68957436f55d67f0"}
Oct 30 18:48:06.456157 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:06.455733    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8" event={"ID":"287c6859-11cf-4a66-a3f8-b866c5477bb1","Type":"ContainerStarted","Data":"236b7d2c484adf9093da42a41e5295bad70c3f57d96e2aa638985cfbc8dab2eb"}
Oct 30 18:48:06.710072 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:06.709968    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-76b58b9cbb-gctk8" podStartSLOduration=1.995605368 podStartE2EDuration="4.709955567s" totalImagesPullingTime="2.714350199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:06.663766171 +0000 UTC m=+59.609934073" watchObservedRunningTime="2026-10-30 18:48:06.709955567 +0000 UTC m=+59.656123466"
Oct 30 18:48:07.108133 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.108104    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-66ff855b85-xwnns"]
Oct 30 18:48:07.110661 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.110639    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.159523 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.159504    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Oct 30 18:48:07.159622 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.159542    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Oct 30 18:48:07.159622 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.159613    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-lk2tt\""
Oct 30 18:48:07.171612 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.171590    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66ff855b85-xwnns"]
Oct 30 18:48:07.202347 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.202328    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-serving-cert\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.202453 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.202361    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-trusted-ca-bundle\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.202453 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.202383    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-oauth-config\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.202453 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.202398    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vfb7\" (UniqueName: \"kubernetes.io/projected/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-kube-api-access-5vfb7\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.202550 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.202492    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-service-ca\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.202550 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.202529    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-oauth-serving-cert\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.202612 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.202574    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-config\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.202642 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.202613    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-tmp\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.226045 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.225991    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Oct 30 18:48:07.226194 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.226051    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Oct 30 18:48:07.226194 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.226061    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Oct 30 18:48:07.226194 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.226067    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Oct 30 18:48:07.226194 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.226168    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Oct 30 18:48:07.303008 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.302990    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-serving-cert\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.303106 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.303023    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-trusted-ca-bundle\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.303106 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.303062    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-oauth-config\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.303106 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.303085    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vfb7\" (UniqueName: \"kubernetes.io/projected/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-kube-api-access-5vfb7\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.303215 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.303117    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-service-ca\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.303215 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.303154    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-oauth-serving-cert\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.303316 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.303297    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-config\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.303711 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.303686    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-tmp\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.304288 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.304261    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-service-ca\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.304423 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.304285    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-oauth-serving-cert\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.304516 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.304325    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-tmp\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.304799 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.304774    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-config\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.306216 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.306191    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-serving-cert\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.309694 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.309674    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-oauth-config\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.311912 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.311895    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Oct 30 18:48:07.314622 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.314598    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-trusted-ca-bundle\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.361023 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.360974    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vfb7\" (UniqueName: \"kubernetes.io/projected/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-kube-api-access-5vfb7\") pod \"console-66ff855b85-xwnns\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") " pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.420663 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.420642    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:07.421101 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.421072    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:07.787559 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.787526    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66ff855b85-xwnns"]
Oct 30 18:48:07.795342 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:48:07.795313    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d70d12f_a3ce_4371_9fd6_e2b68523d2bb.slice/crio-2bd0cde2816297d27bd94c52c914f5eebe03fc5beb2b213deeb17f75c8d63c13 WatchSource:0}: Error finding container 2bd0cde2816297d27bd94c52c914f5eebe03fc5beb2b213deeb17f75c8d63c13: Status 404 returned error can't find the container with id 2bd0cde2816297d27bd94c52c914f5eebe03fc5beb2b213deeb17f75c8d63c13
Oct 30 18:48:07.836895 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:07.836872    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:08.019021 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:08.018987    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:08.465655 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:08.465617    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66ff855b85-xwnns" event={"ID":"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb","Type":"ContainerStarted","Data":"e4b59d8da07bf6c77e8913d6d8931d53d3b82e639f5e121882b13fc6b8c96e48"}
Oct 30 18:48:08.465814 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:08.465679    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66ff855b85-xwnns" event={"ID":"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb","Type":"ContainerStarted","Data":"2bd0cde2816297d27bd94c52c914f5eebe03fc5beb2b213deeb17f75c8d63c13"}
Oct 30 18:48:08.576759 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:08.576714    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-66ff855b85-xwnns" podStartSLOduration=1.576701557 podStartE2EDuration="1.576701557s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:08.576414402 +0000 UTC m=+61.522582318" watchObservedRunningTime="2026-10-30 18:48:08.576701557 +0000 UTC m=+61.522869459"
Oct 30 18:48:11.630667 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:11.630631    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:17.421879 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:17.421848    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:17.422274 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:17.421896    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:17.426409 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:17.426390    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:17.503152 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:17.503132    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:48:27.549414 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:27.549375    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:48:33.381346 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:48:33.381310    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:48:54.774814 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:48:54.774786    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:03.381019 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:49:03.380981    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:49:14.705563 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:14.705530    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:17.651392 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.651355    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console/console-7648585d6-2bv69"]
Oct 30 18:49:17.654589 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.654568    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.757714 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.757691    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7648585d6-2bv69"]
Oct 30 18:49:17.806316 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.806282    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-console-config\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.806460 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.806331    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5v8t\" (UniqueName: \"kubernetes.io/projected/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-kube-api-access-g5v8t\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.806460 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.806354    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-tmp\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.806553 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.806467    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-oauth-serving-cert\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.806553 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.806493    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-service-ca\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.806553 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.806516    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-console-oauth-config\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.806668 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.806557    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-console-serving-cert\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.806668 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.806604    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-trusted-ca-bundle\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.907468 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.907403    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-console-oauth-config\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.907468 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.907448    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-console-serving-cert\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.907650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.907477    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-trusted-ca-bundle\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.907650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.907521    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-console-config\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.907650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.907566    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5v8t\" (UniqueName: \"kubernetes.io/projected/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-kube-api-access-g5v8t\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.907650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.907589    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-tmp\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.907650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.907618    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-oauth-serving-cert\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.907650 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.907644    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-service-ca\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.908125 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.908063    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-tmp\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.908381 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.908352    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-console-config\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.908465 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.908439    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-service-ca\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.908576 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.908558    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-trusted-ca-bundle\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.908638 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.908619    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-oauth-serving-cert\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.909815 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.909800    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-console-oauth-config\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.910011 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.909996    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-console-serving-cert\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.964638 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.964620    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5v8t\" (UniqueName: \"kubernetes.io/projected/a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7-kube-api-access-g5v8t\") pod \"console-7648585d6-2bv69\" (UID: \"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7\") " pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.965416 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.965395    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:17.972679 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:17.972643    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:18.205616 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:18.205527    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:18.386671 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:18.386642    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7648585d6-2bv69"]
Oct 30 18:49:18.395812 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:49:18.395788    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda51bfbd3_9914_4a7a_a3b3_8017a1ad8ca7.slice/crio-673c397506db9cc6d5496c7787570abcf4c35297623c0babff927b18b8852971 WatchSource:0}: Error finding container 673c397506db9cc6d5496c7787570abcf4c35297623c0babff927b18b8852971: Status 404 returned error can't find the container with id 673c397506db9cc6d5496c7787570abcf4c35297623c0babff927b18b8852971
Oct 30 18:49:18.714743 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:18.714709    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7648585d6-2bv69" event={"ID":"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7","Type":"ContainerStarted","Data":"b34b50e2ab1d14a4a3b7aa8fc71c7da35c959e105277870c2d4d55a9da72dc1d"}
Oct 30 18:49:18.715128 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:18.714751    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7648585d6-2bv69" event={"ID":"a51bfbd3-9914-4a7a-a3b3-8017a1ad8ca7","Type":"ContainerStarted","Data":"673c397506db9cc6d5496c7787570abcf4c35297623c0babff927b18b8852971"}
Oct 30 18:49:18.836163 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:18.836119    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console/console-7648585d6-2bv69" podStartSLOduration=1.83610606 podStartE2EDuration="1.83610606s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:49:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:18.835110478 +0000 UTC m=+131.781278376" watchObservedRunningTime="2026-10-30 18:49:18.83610606 +0000 UTC m=+131.782273962"
Oct 30 18:49:19.357062 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:19.357010    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:27.429934 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:27.429902    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:27.657907 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:27.657868    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:27.966129 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:27.966102    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:27.966279 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:27.966160    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:27.970719 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:27.970696    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:28.752291 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:28.752264    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7648585d6-2bv69"
Oct 30 18:49:28.911610 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:28.911586    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66ff855b85-xwnns"]
Oct 30 18:49:33.381967 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:49:33.381935    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:49:40.312084 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:40.312049    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:40.492206 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:40.492174    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:53.930901 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:53.930830    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-console/console-66ff855b85-xwnns" podUID="6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" containerName="console" containerID="cri-o://e4b59d8da07bf6c77e8913d6d8931d53d3b82e639f5e121882b13fc6b8c96e48" gracePeriod=15
Oct 30 18:49:54.171759 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.171738    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:49:54.270082 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.270060    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-serving-cert\") pod \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") "
Oct 30 18:49:54.270180 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.270095    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-oauth-serving-cert\") pod \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") "
Oct 30 18:49:54.270180 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.270142    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-trusted-ca-bundle\") pod \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") "
Oct 30 18:49:54.270256 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.270178    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-service-ca\") pod \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") "
Oct 30 18:49:54.270256 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.270241    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-tmp\") pod \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") "
Oct 30 18:49:54.270331 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.270278    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-oauth-config\") pod \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") "
Oct 30 18:49:54.270331 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.270320    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-kube-api-access-5vfb7\" (UniqueName: \"kubernetes.io/projected/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-kube-api-access-5vfb7\") pod \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") "
Oct 30 18:49:54.270430 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.270352    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-config\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-config\") pod \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\" (UID: \"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb\") "
Oct 30 18:49:54.270621 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.270589    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-tmp" pod "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" (UID: "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb"). InnerVolumeSpecName "tmp". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:49:54.270718 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.270659    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-oauth-serving-cert" pod "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" (UID: "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:49:54.270718 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.270665    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-trusted-ca-bundle" pod "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" (UID: "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:49:54.270718 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.270686    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-service-ca" pod "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" (UID: "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:49:54.270917 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.270889    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-config" pod "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" (UID: "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:49:54.272239 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.272215    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-serving-cert" pod "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" (UID: "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:49:54.272336 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.272315    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-oauth-config" pod "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" (UID: "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:49:54.272522 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.272494    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-kube-api-access-5vfb7" pod "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" (UID: "6d70d12f-a3ce-4371-9fd6-e2b68523d2bb"). InnerVolumeSpecName "kube-api-access-5vfb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:49:54.371489 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.371466    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vfb7\" (UniqueName: \"kubernetes.io/projected/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-kube-api-access-5vfb7\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:49:54.371489 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.371488    5704 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-config\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:49:54.371653 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.371498    5704 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-serving-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:49:54.371653 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.371509    5704 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-oauth-serving-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:49:54.371653 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.371518    5704 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-trusted-ca-bundle\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:49:54.371653 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.371526    5704 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:49:54.371653 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.371534    5704 reconciler_common.go:299] "Volume detached for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-tmp\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:49:54.371653 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.371543    5704 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb-console-oauth-config\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:49:54.837522 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.837490    5704 generic.go:356] "Generic (PLEG): container finished" podID="6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" containerID="e4b59d8da07bf6c77e8913d6d8931d53d3b82e639f5e121882b13fc6b8c96e48" exitCode=0
Oct 30 18:49:54.837665 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.837536    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66ff855b85-xwnns"
Oct 30 18:49:54.837665 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.837563    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66ff855b85-xwnns" event={"ID":"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb","Type":"ContainerDied","Data":"e4b59d8da07bf6c77e8913d6d8931d53d3b82e639f5e121882b13fc6b8c96e48"}
Oct 30 18:49:54.837665 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.837600    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66ff855b85-xwnns" event={"ID":"6d70d12f-a3ce-4371-9fd6-e2b68523d2bb","Type":"ContainerDied","Data":"2bd0cde2816297d27bd94c52c914f5eebe03fc5beb2b213deeb17f75c8d63c13"}
Oct 30 18:49:54.837665 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.837618    5704 scope.go:122] "RemoveContainer" containerID="e4b59d8da07bf6c77e8913d6d8931d53d3b82e639f5e121882b13fc6b8c96e48"
Oct 30 18:49:54.848388 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.848366    5704 scope.go:122] "RemoveContainer" containerID="e4b59d8da07bf6c77e8913d6d8931d53d3b82e639f5e121882b13fc6b8c96e48"
Oct 30 18:49:54.848846 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:49:54.848817    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4b59d8da07bf6c77e8913d6d8931d53d3b82e639f5e121882b13fc6b8c96e48\": container with ID starting with e4b59d8da07bf6c77e8913d6d8931d53d3b82e639f5e121882b13fc6b8c96e48 not found: ID does not exist" containerID="e4b59d8da07bf6c77e8913d6d8931d53d3b82e639f5e121882b13fc6b8c96e48"
Oct 30 18:49:54.848951 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.848859    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4b59d8da07bf6c77e8913d6d8931d53d3b82e639f5e121882b13fc6b8c96e48"} err="failed to get container status \"e4b59d8da07bf6c77e8913d6d8931d53d3b82e639f5e121882b13fc6b8c96e48\": rpc error: code = NotFound desc = could not find container \"e4b59d8da07bf6c77e8913d6d8931d53d3b82e639f5e121882b13fc6b8c96e48\": container with ID starting with e4b59d8da07bf6c77e8913d6d8931d53d3b82e639f5e121882b13fc6b8c96e48 not found: ID does not exist"
Oct 30 18:49:54.993176 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:54.993142    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66ff855b85-xwnns"]
Oct 30 18:49:55.039086 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:55.039058    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66ff855b85-xwnns"]
Oct 30 18:49:55.094565 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:55.094498    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:49:55.176739 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:49:55.176708    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" path="/var/lib/kubelet/pods/6d70d12f-a3ce-4371-9fd6-e2b68523d2bb/volumes"
Oct 30 18:50:03.380421 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:50:03.380382    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:50:22.320489 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:50:22.320408    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:33.380948 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:50:33.380909    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:50:39.592306 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:50:39.592260    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:39.821999 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:50:39.821967    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:50:55.439979 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:50:55.439951    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:03.380914 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:51:03.380877    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:51:33.380689 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:51:33.380651    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:51:39.528774 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:39.528740    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:44.900566 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:44.900532    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:45.136400 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.136355    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:45.786447 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.786419    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"]
Oct 30 18:51:45.786832 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.786815    5704 cpu_manager.go:414] "containerMap: removing container" podUID="6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" containerName="console"
Oct 30 18:51:45.786932 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.786835    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" containerName="console"
Oct 30 18:51:45.786932 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.786911    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d70d12f-a3ce-4371-9fd6-e2b68523d2bb" containerName="console"
Oct 30 18:51:45.789999 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.789977    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"
Oct 30 18:51:45.843637 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.843612    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Oct 30 18:51:45.861982 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.861954    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts\" (UID: \"7fdbae3b-13da-401c-b495-88300357a244\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"
Oct 30 18:51:45.862097 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.861987    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts\" (UID: \"7fdbae3b-13da-401c-b495-88300357a244\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"
Oct 30 18:51:45.862141 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.862125    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnbw8\" (UniqueName: \"kubernetes.io/projected/7fdbae3b-13da-401c-b495-88300357a244-kube-api-access-qnbw8\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts\" (UID: \"7fdbae3b-13da-401c-b495-88300357a244\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"
Oct 30 18:51:45.888182 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.888160    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Oct 30 18:51:45.888182 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.888165    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"c7537007944577f64df34960ace3c69136c416-139ec370-dockercfg-xtspv\""
Oct 30 18:51:45.896749 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.896722    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"]
Oct 30 18:51:45.962792 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.962767    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts\" (UID: \"7fdbae3b-13da-401c-b495-88300357a244\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"
Oct 30 18:51:45.962792 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.962793    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts\" (UID: \"7fdbae3b-13da-401c-b495-88300357a244\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"
Oct 30 18:51:45.963164 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.962930    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnbw8\" (UniqueName: \"kubernetes.io/projected/7fdbae3b-13da-401c-b495-88300357a244-kube-api-access-qnbw8\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts\" (UID: \"7fdbae3b-13da-401c-b495-88300357a244\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"
Oct 30 18:51:45.963205 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.963187    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-util\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts\" (UID: \"7fdbae3b-13da-401c-b495-88300357a244\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"
Oct 30 18:51:45.963239 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:45.963203    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-bundle\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts\" (UID: \"7fdbae3b-13da-401c-b495-88300357a244\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"
Oct 30 18:51:46.020341 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.020315    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnbw8\" (UniqueName: \"kubernetes.io/projected/7fdbae3b-13da-401c-b495-88300357a244-kube-api-access-qnbw8\") pod \"c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts\" (UID: \"7fdbae3b-13da-401c-b495-88300357a244\") " pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"
Oct 30 18:51:46.105267 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.105215    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"
Oct 30 18:51:46.438133 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.438059    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"]
Oct 30 18:51:46.441498 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.441467    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.473816 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.473796    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"]
Oct 30 18:51:46.476190 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.476165    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:46.481974 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:51:46.481951    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fdbae3b_13da_401c_b495_88300357a244.slice/crio-b1c33b847d770a780310588a19b9ad1983356f9b1efd13da610319be6448011d WatchSource:0}: Error finding container b1c33b847d770a780310588a19b9ad1983356f9b1efd13da610319be6448011d: Status 404 returned error can't find the container with id b1c33b847d770a780310588a19b9ad1983356f9b1efd13da610319be6448011d
Oct 30 18:51:46.493189 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.493169    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Oct 30 18:51:46.493277 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.493222    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Oct 30 18:51:46.493277 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.493251    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Oct 30 18:51:46.493277 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.493259    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Oct 30 18:51:46.493277 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.493258    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Oct 30 18:51:46.493424 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.493287    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Oct 30 18:51:46.493730 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.493714    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-4r545\""
Oct 30 18:51:46.520117 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.520095    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"]
Oct 30 18:51:46.567676 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.567650    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.567778 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.567687    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.567821 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.567784    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.567821 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.567811    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb7af7e1-435e-4996-8474-4f0144d33721-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.567909 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.567828    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwrr4\" (UniqueName: \"kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-kube-api-access-hwrr4\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.567909 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.567847    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.567987 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.567920    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.648661 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.648639    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:46.668644 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.668623    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.668724 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.668674    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.668724 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.668694    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.668793 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.668739    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.668793 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.668763    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb7af7e1-435e-4996-8474-4f0144d33721-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.668793 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.668785    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwrr4\" (UniqueName: \"kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-kube-api-access-hwrr4\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.668934 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.668810    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.669348 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.669324    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.671093 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.671074    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb7af7e1-435e-4996-8474-4f0144d33721-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.671210 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.671192    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.671288 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.671272    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.671344 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.671295    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.725974 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.725922    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.726200 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.726182    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwrr4\" (UniqueName: \"kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-kube-api-access-hwrr4\") pod \"istiod-openshift-gateway-7876bfcb74-884p7\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:46.752962 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:46.752918    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:47.176736 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:47.176709    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"]
Oct 30 18:51:47.182731 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:51:47.182700    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb7af7e1_435e_4996_8474_4f0144d33721.slice/crio-0ce3937401e680f9b57130f4e0615fade2b82f144eff7e84d331023dc1d00a1b WatchSource:0}: Error finding container 0ce3937401e680f9b57130f4e0615fade2b82f144eff7e84d331023dc1d00a1b: Status 404 returned error can't find the container with id 0ce3937401e680f9b57130f4e0615fade2b82f144eff7e84d331023dc1d00a1b
Oct 30 18:51:47.217958 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:47.217931    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts" event={"ID":"7fdbae3b-13da-401c-b495-88300357a244","Type":"ContainerStarted","Data":"b1c33b847d770a780310588a19b9ad1983356f9b1efd13da610319be6448011d"}
Oct 30 18:51:47.219142 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:47.219119    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7" event={"ID":"cb7af7e1-435e-4996-8474-4f0144d33721","Type":"ContainerStarted","Data":"0ce3937401e680f9b57130f4e0615fade2b82f144eff7e84d331023dc1d00a1b"}
Oct 30 18:51:47.895094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:47.894882    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:51.045057 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:51.045006    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:51:51.045479 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:51.045108    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:51:51.244221 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:51.244172    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7" event={"ID":"cb7af7e1-435e-4996-8474-4f0144d33721","Type":"ContainerStarted","Data":"6e1820a6cae2cd560c74cc49638972fa7ca4261b834b63b5d81d1b9b1f77e68b"}
Oct 30 18:51:51.244386 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:51.244289    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:51.451125 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:51.450992    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7" podStartSLOduration=1.5914241740000001 podStartE2EDuration="5.450973369s" totalImagesPullingTime="3.859549195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:51.404129881 +0000 UTC m=+284.350297784" watchObservedRunningTime="2026-10-30 18:51:51.450973369 +0000 UTC m=+284.397141273"
Oct 30 18:51:52.251256 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:52.251220    5704 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-884p7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 18:51:52.251718 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:52.251280    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7" podUID="cb7af7e1-435e-4996-8474-4f0144d33721" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 18:51:52.461651 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:52.461615    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:52.639549 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:52.639465    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:53.253560 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:53.253530    5704 generic.go:356] "Generic (PLEG): container finished" podID="7fdbae3b-13da-401c-b495-88300357a244" containerID="18c7bc857609ee76529eb12c240f281b28f1cf0bddfb50b22183f7f309978cf3" exitCode=0
Oct 30 18:51:53.253937 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:53.253616    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts" event={"ID":"7fdbae3b-13da-401c-b495-88300357a244","Type":"ContainerDied","Data":"18c7bc857609ee76529eb12c240f281b28f1cf0bddfb50b22183f7f309978cf3"}
Oct 30 18:51:53.442319 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:53.442281    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:53.775842 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:53.775812    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:53.947339 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:53.947310    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:55.250718 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:55.250689    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:51:55.779956 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:55.779924    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:51:57.270243 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:57.270214    5704 generic.go:356] "Generic (PLEG): container finished" podID="7fdbae3b-13da-401c-b495-88300357a244" containerID="1a1426008ad9bd3d9e050965c69d4dca1e70e1354d43f7d0e2d7634afee7a1da" exitCode=0
Oct 30 18:51:57.270604 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:57.270296    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts" event={"ID":"7fdbae3b-13da-401c-b495-88300357a244","Type":"ContainerDied","Data":"1a1426008ad9bd3d9e050965c69d4dca1e70e1354d43f7d0e2d7634afee7a1da"}
Oct 30 18:51:57.629631 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:51:57.629596    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:03.383237 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:52:03.383199    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:52:04.307631 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:04.307550    5704 generic.go:356] "Generic (PLEG): container finished" podID="7fdbae3b-13da-401c-b495-88300357a244" containerID="1e03fa30adf451a9586f8231a59fde94a8ede73de345dffb12ab6bd7c3683a73" exitCode=0
Oct 30 18:52:04.307788 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:04.307633    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts" event={"ID":"7fdbae3b-13da-401c-b495-88300357a244","Type":"ContainerDied","Data":"1e03fa30adf451a9586f8231a59fde94a8ede73de345dffb12ab6bd7c3683a73"}
Oct 30 18:52:05.429893 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:05.429872    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"
Oct 30 18:52:05.439639 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:05.439617    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7fdbae3b-13da-401c-b495-88300357a244-kube-api-access-qnbw8\" (UniqueName: \"kubernetes.io/projected/7fdbae3b-13da-401c-b495-88300357a244-kube-api-access-qnbw8\") pod \"7fdbae3b-13da-401c-b495-88300357a244\" (UID: \"7fdbae3b-13da-401c-b495-88300357a244\") "
Oct 30 18:52:05.439758 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:05.439672    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-util\" (UniqueName: \"kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-util\") pod \"7fdbae3b-13da-401c-b495-88300357a244\" (UID: \"7fdbae3b-13da-401c-b495-88300357a244\") "
Oct 30 18:52:05.439758 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:05.439695    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-bundle\" (UniqueName: \"kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-bundle\") pod \"7fdbae3b-13da-401c-b495-88300357a244\" (UID: \"7fdbae3b-13da-401c-b495-88300357a244\") "
Oct 30 18:52:05.440300 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:05.440275    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-bundle" pod "7fdbae3b-13da-401c-b495-88300357a244" (UID: "7fdbae3b-13da-401c-b495-88300357a244"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:05.441865 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:05.441837    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fdbae3b-13da-401c-b495-88300357a244-kube-api-access-qnbw8" pod "7fdbae3b-13da-401c-b495-88300357a244" (UID: "7fdbae3b-13da-401c-b495-88300357a244"). InnerVolumeSpecName "kube-api-access-qnbw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:52:05.447566 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:05.447539    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-util" pod "7fdbae3b-13da-401c-b495-88300357a244" (UID: "7fdbae3b-13da-401c-b495-88300357a244"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:52:05.540393 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:05.540371    5704 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-util\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:05.540498 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:05.540395    5704 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7fdbae3b-13da-401c-b495-88300357a244-bundle\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:05.540498 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:05.540409    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnbw8\" (UniqueName: \"kubernetes.io/projected/7fdbae3b-13da-401c-b495-88300357a244-kube-api-access-qnbw8\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:52:06.316781 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:06.316756    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts"
Oct 30 18:52:06.316781 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:06.316772    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c7537007944577f64df34960ace3c69136c4162cd56d2c48c7c51e651cdr8ts" event={"ID":"7fdbae3b-13da-401c-b495-88300357a244","Type":"ContainerDied","Data":"b1c33b847d770a780310588a19b9ad1983356f9b1efd13da610319be6448011d"}
Oct 30 18:52:06.317050 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:06.316812    5704 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b1c33b847d770a780310588a19b9ad1983356f9b1efd13da610319be6448011d"
Oct 30 18:52:08.296985 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:08.296942    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:08.475861 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:08.475827    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:33.382674 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:52:33.382637    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:52:35.963787 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:35.963751    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:36.148995 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:36.148959    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:36.848250 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:36.848214    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:37.034566 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.034522    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:37.077620 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.077591    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"]
Oct 30 18:52:37.077909 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.077897    5704 cpu_manager.go:414] "containerMap: removing container" podUID="7fdbae3b-13da-401c-b495-88300357a244" containerName="util"
Oct 30 18:52:37.077961 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.077909    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7fdbae3b-13da-401c-b495-88300357a244" containerName="util"
Oct 30 18:52:37.077961 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.077917    5704 cpu_manager.go:414] "containerMap: removing container" podUID="7fdbae3b-13da-401c-b495-88300357a244" containerName="pull"
Oct 30 18:52:37.077961 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.077922    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7fdbae3b-13da-401c-b495-88300357a244" containerName="pull"
Oct 30 18:52:37.077961 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.077930    5704 cpu_manager.go:414] "containerMap: removing container" podUID="7fdbae3b-13da-401c-b495-88300357a244" containerName="extract"
Oct 30 18:52:37.077961 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.077936    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7fdbae3b-13da-401c-b495-88300357a244" containerName="extract"
Oct 30 18:52:37.078166 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.078008    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="7fdbae3b-13da-401c-b495-88300357a244" containerName="extract"
Oct 30 18:52:37.080352 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.080337    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.180558 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.180494    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-metrics-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.180558 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.180525    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-manifests\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.180558 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.180549    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-tmp\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.180753 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.180617    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.180753 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.180678    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgqns\" (UniqueName: \"kubernetes.io/projected/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-kube-api-access-vgqns\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.180753 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.180718    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.205856 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.205830    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\""
Oct 30 18:52:37.205986 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.205877    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\""
Oct 30 18:52:37.205986 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.205902    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-wxrjr\""
Oct 30 18:52:37.205986 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.205908    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\""
Oct 30 18:52:37.205986 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.205912    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\""
Oct 30 18:52:37.205986 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.205928    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\""
Oct 30 18:52:37.281718 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.281691    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.281851 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.281743    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-metrics-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.281851 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.281769    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-manifests\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.281851 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.281802    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-tmp\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.281851 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.281828    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.282080 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.281856    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgqns\" (UniqueName: \"kubernetes.io/projected/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-kube-api-access-vgqns\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.282205 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.282184    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-manifests\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.284239 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.284213    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-tmp\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.284351 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.284261    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.284351 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.284334    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-metrics-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.284351 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.284341    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-webhook-cert\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.305284 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.305263    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:37.335531 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.335503    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"]
Oct 30 18:52:37.339640 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.339618    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgqns\" (UniqueName: \"kubernetes.io/projected/d714fdb7-13f7-4ffe-9d27-fd8634b831d0-kube-api-access-vgqns\") pod \"opendatahub-operator-controller-manager-575fc98dd-v7h8q\" (UID: \"d714fdb7-13f7-4ffe-9d27-fd8634b831d0\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.393298 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.393276    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:37.478337 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.478312    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:37.660009 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.659979    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:37.883346 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:37.883309    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"]
Oct 30 18:52:37.891319 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:52:37.891293    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd714fdb7_13f7_4ffe_9d27_fd8634b831d0.slice/crio-c931ae8738bbc691453bd8917e6938c95d51cbd4f98d45998e66b4dcbc635dcd WatchSource:0}: Error finding container c931ae8738bbc691453bd8917e6938c95d51cbd4f98d45998e66b4dcbc635dcd: Status 404 returned error can't find the container with id c931ae8738bbc691453bd8917e6938c95d51cbd4f98d45998e66b4dcbc635dcd
Oct 30 18:52:38.432562 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:38.432526    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q" event={"ID":"d714fdb7-13f7-4ffe-9d27-fd8634b831d0","Type":"ContainerStarted","Data":"c931ae8738bbc691453bd8917e6938c95d51cbd4f98d45998e66b4dcbc635dcd"}
Oct 30 18:52:39.710269 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:39.710219    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:40.566310 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:40.566279    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:40.806286 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:40.806254    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:40.983088 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:40.982974    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:42.452852 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:42.452821    5704 generic.go:356] "Generic (PLEG): container finished" podID="d714fdb7-13f7-4ffe-9d27-fd8634b831d0" containerID="0f627fb45e153bb499739de03b0cc1f7c5f3691f48bf383ba6fab81da3b541b8" exitCode=0
Oct 30 18:52:42.453279 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:42.452894    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q" event={"ID":"d714fdb7-13f7-4ffe-9d27-fd8634b831d0","Type":"ContainerDied","Data":"0f627fb45e153bb499739de03b0cc1f7c5f3691f48bf383ba6fab81da3b541b8"}
Oct 30 18:52:43.459723 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:43.459687    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q" event={"ID":"d714fdb7-13f7-4ffe-9d27-fd8634b831d0","Type":"ContainerStarted","Data":"b4f209c3286eeadef7fd7c8557e1b0363a9b06ca6aa62785422c4081dfe644f5"}
Oct 30 18:52:43.460223 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:43.460122    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:43.700782 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:43.700672    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q" podStartSLOduration=2.885692761 podStartE2EDuration="7.700656508s" totalImagesPullingTime="4.374454796s" totalInitContainerRuntime="440.508951ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:43.654242805 +0000 UTC m=+336.600410708" watchObservedRunningTime="2026-10-30 18:52:43.700656508 +0000 UTC m=+336.646824402"
Oct 30 18:52:44.132871 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:44.132837    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:44.374622 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:44.374589    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:46.307023 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:46.306989    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:47.160225 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:47.160188    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:48.113822 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:48.113710    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:49.783643 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:49.783613    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:50.009482 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:50.009454    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:54.318996 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:54.318960    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:54.466644 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:54.466620    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-575fc98dd-v7h8q"
Oct 30 18:52:54.968295 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:54.968257    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:55.169452 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:55.169420    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:55.398310 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:55.398278    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:56.105551 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:56.105421    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:57.353146 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:57.353112    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:52:59.101499 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:52:59.101460    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:01.798936 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:01.798898    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:03.381561 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:53:03.381516    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:53:03.538514 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:03.538481    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:04.125819 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:04.125787    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:04.366708 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:04.366671    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:04.807165 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:04.807123    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:05.000159 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:05.000126    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:05.678230 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:05.678192    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:05.901198 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:05.901165    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:06.771296 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:06.771257    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:07.474093 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:07.474056    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:10.918088 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:10.918053    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:11.154180 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:11.154147    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:11.710913 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:11.710886    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:11.895490 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:11.895460    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:12.143591 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.143559    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:12.325294 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.325263    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:12.597016 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.596984    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-k6957"]
Oct 30 18:53:12.600813 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.600790    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k6957"
Oct 30 18:53:12.652791 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.652766    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Oct 30 18:53:12.670281 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.670256    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfmmz\" (UniqueName: \"kubernetes.io/projected/2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4-kube-api-access-kfmmz\") pod \"cert-manager-webhook-5dbbbf75cc-k6957\" (UID: \"2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k6957"
Oct 30 18:53:12.670374 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.670308    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-k6957\" (UID: \"2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k6957"
Oct 30 18:53:12.698618 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.698597    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Oct 30 18:53:12.698723 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.698709    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-sw8cb\""
Oct 30 18:53:12.699505 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.699486    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-k6957"]
Oct 30 18:53:12.771606 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.771583    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfmmz\" (UniqueName: \"kubernetes.io/projected/2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4-kube-api-access-kfmmz\") pod \"cert-manager-webhook-5dbbbf75cc-k6957\" (UID: \"2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k6957"
Oct 30 18:53:12.771757 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.771632    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-k6957\" (UID: \"2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k6957"
Oct 30 18:53:12.829439 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.829408    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-k6957\" (UID: \"2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k6957"
Oct 30 18:53:12.829760 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.829739    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfmmz\" (UniqueName: \"kubernetes.io/projected/2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4-kube-api-access-kfmmz\") pod \"cert-manager-webhook-5dbbbf75cc-k6957\" (UID: \"2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k6957"
Oct 30 18:53:12.921912 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:12.921851    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k6957"
Oct 30 18:53:13.287106 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:13.287079    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-k6957"]
Oct 30 18:53:13.296048 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:53:13.296010    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2c8b6e9d_25c3_47ed_9ae2_95af47bd69d4.slice/crio-79646c6b01b8fa279db93b1b88e56aa14578ab752ecef92e4bf04ee64d44ef67 WatchSource:0}: Error finding container 79646c6b01b8fa279db93b1b88e56aa14578ab752ecef92e4bf04ee64d44ef67: Status 404 returned error can't find the container with id 79646c6b01b8fa279db93b1b88e56aa14578ab752ecef92e4bf04ee64d44ef67
Oct 30 18:53:13.298019 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:13.298003    5704 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:53:13.572611 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:13.572543    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k6957" event={"ID":"2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4","Type":"ContainerStarted","Data":"79646c6b01b8fa279db93b1b88e56aa14578ab752ecef92e4bf04ee64d44ef67"}
Oct 30 18:53:15.804251 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:15.804218    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:16.094382 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:16.094296    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-kffdx"]
Oct 30 18:53:16.097365 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:16.097340    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-kffdx"
Oct 30 18:53:16.146347 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:16.146319    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-sw7qb\""
Oct 30 18:53:16.156320 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:16.156292    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-kffdx"]
Oct 30 18:53:16.199799 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:16.199766    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2fdw\" (UniqueName: \"kubernetes.io/projected/99a59b42-fe6f-43f4-8644-f9ab1082f9a8-kube-api-access-b2fdw\") pod \"cert-manager-597b95cb84-kffdx\" (UID: \"99a59b42-fe6f-43f4-8644-f9ab1082f9a8\") " pod="cert-manager/cert-manager-597b95cb84-kffdx"
Oct 30 18:53:16.199940 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:16.199903    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99a59b42-fe6f-43f4-8644-f9ab1082f9a8-bound-sa-token\") pod \"cert-manager-597b95cb84-kffdx\" (UID: \"99a59b42-fe6f-43f4-8644-f9ab1082f9a8\") " pod="cert-manager/cert-manager-597b95cb84-kffdx"
Oct 30 18:53:16.301148 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:16.301109    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99a59b42-fe6f-43f4-8644-f9ab1082f9a8-bound-sa-token\") pod \"cert-manager-597b95cb84-kffdx\" (UID: \"99a59b42-fe6f-43f4-8644-f9ab1082f9a8\") " pod="cert-manager/cert-manager-597b95cb84-kffdx"
Oct 30 18:53:16.301313 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:16.301191    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2fdw\" (UniqueName: \"kubernetes.io/projected/99a59b42-fe6f-43f4-8644-f9ab1082f9a8-kube-api-access-b2fdw\") pod \"cert-manager-597b95cb84-kffdx\" (UID: \"99a59b42-fe6f-43f4-8644-f9ab1082f9a8\") " pod="cert-manager/cert-manager-597b95cb84-kffdx"
Oct 30 18:53:16.359791 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:16.359727    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/99a59b42-fe6f-43f4-8644-f9ab1082f9a8-bound-sa-token\") pod \"cert-manager-597b95cb84-kffdx\" (UID: \"99a59b42-fe6f-43f4-8644-f9ab1082f9a8\") " pod="cert-manager/cert-manager-597b95cb84-kffdx"
Oct 30 18:53:16.360072 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:16.360053    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2fdw\" (UniqueName: \"kubernetes.io/projected/99a59b42-fe6f-43f4-8644-f9ab1082f9a8-kube-api-access-b2fdw\") pod \"cert-manager-597b95cb84-kffdx\" (UID: \"99a59b42-fe6f-43f4-8644-f9ab1082f9a8\") " pod="cert-manager/cert-manager-597b95cb84-kffdx"
Oct 30 18:53:16.411083 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:16.411054    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-kffdx"
Oct 30 18:53:16.785786 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:16.785754    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-kffdx"]
Oct 30 18:53:16.794995 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:53:16.794968    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99a59b42_fe6f_43f4_8644_f9ab1082f9a8.slice/crio-1a26491955aabc7a175e5b389d87c1484ef4307c21bd49c42e93126ad23e55c4 WatchSource:0}: Error finding container 1a26491955aabc7a175e5b389d87c1484ef4307c21bd49c42e93126ad23e55c4: Status 404 returned error can't find the container with id 1a26491955aabc7a175e5b389d87c1484ef4307c21bd49c42e93126ad23e55c4
Oct 30 18:53:17.465688 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:17.465656    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:17.590766 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:17.590734    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k6957" event={"ID":"2c8b6e9d-25c3-47ed-9ae2-95af47bd69d4","Type":"ContainerStarted","Data":"e2613d79601269cc80bda93611f4783033a7e57a18d1ba2b9198ca292a472660"}
Oct 30 18:53:17.590943 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:17.590819    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k6957"
Oct 30 18:53:17.592077 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:17.592052    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-kffdx" event={"ID":"99a59b42-fe6f-43f4-8644-f9ab1082f9a8","Type":"ContainerStarted","Data":"1a26491955aabc7a175e5b389d87c1484ef4307c21bd49c42e93126ad23e55c4"}
Oct 30 18:53:17.657138 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:17.657110    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:17.700665 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:17.700618    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k6957" podStartSLOduration=2.049419585 podStartE2EDuration="5.700606297s" totalImagesPullingTime="3.651186712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:17.700242564 +0000 UTC m=+370.646410479" watchObservedRunningTime="2026-10-30 18:53:17.700606297 +0000 UTC m=+370.646774197"
Oct 30 18:53:17.848935 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:17.848900    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:18.490396 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:18.490286    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:18.597871 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:18.597824    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-kffdx" event={"ID":"99a59b42-fe6f-43f4-8644-f9ab1082f9a8","Type":"ContainerStarted","Data":"a5b4919d1f0936a29797d998ce652162e8248cd725ca3b10b7e2a3b981c38db6"}
Oct 30 18:53:18.715538 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:18.715488    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-kffdx" podStartSLOduration=1.434926761 podStartE2EDuration="2.715473292s" totalImagesPullingTime="1.280546531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:18.713856695 +0000 UTC m=+371.660024595" watchObservedRunningTime="2026-10-30 18:53:18.715473292 +0000 UTC m=+371.661641195"
Oct 30 18:53:18.735473 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:18.735430    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:19.735763 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:19.735720    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:19.994855 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:19.994773    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:20.070229 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.070199    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"]
Oct 30 18:53:20.073381 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.073362    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"
Oct 30 18:53:20.133667 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.133637    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72ab8598-6730-4763-8a86-4c5ba57c6b65-tls-certs\") pod \"kube-auth-proxy-6df7c6869c-kqhtm\" (UID: \"72ab8598-6730-4763-8a86-4c5ba57c6b65\") " pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"
Oct 30 18:53:20.133790 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.133708    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72ab8598-6730-4763-8a86-4c5ba57c6b65-tmp\") pod \"kube-auth-proxy-6df7c6869c-kqhtm\" (UID: \"72ab8598-6730-4763-8a86-4c5ba57c6b65\") " pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"
Oct 30 18:53:20.133837 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.133785    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf4t9\" (UniqueName: \"kubernetes.io/projected/72ab8598-6730-4763-8a86-4c5ba57c6b65-kube-api-access-nf4t9\") pod \"kube-auth-proxy-6df7c6869c-kqhtm\" (UID: \"72ab8598-6730-4763-8a86-4c5ba57c6b65\") " pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"
Oct 30 18:53:20.169140 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.169114    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Oct 30 18:53:20.169266 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.169175    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Oct 30 18:53:20.169266 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.169182    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vrwzm\""
Oct 30 18:53:20.175005 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.174987    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"]
Oct 30 18:53:20.200706 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.200682    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:20.234797 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.234773    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72ab8598-6730-4763-8a86-4c5ba57c6b65-tls-certs\") pod \"kube-auth-proxy-6df7c6869c-kqhtm\" (UID: \"72ab8598-6730-4763-8a86-4c5ba57c6b65\") " pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"
Oct 30 18:53:20.234907 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.234827    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72ab8598-6730-4763-8a86-4c5ba57c6b65-tmp\") pod \"kube-auth-proxy-6df7c6869c-kqhtm\" (UID: \"72ab8598-6730-4763-8a86-4c5ba57c6b65\") " pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"
Oct 30 18:53:20.234907 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.234866    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf4t9\" (UniqueName: \"kubernetes.io/projected/72ab8598-6730-4763-8a86-4c5ba57c6b65-kube-api-access-nf4t9\") pod \"kube-auth-proxy-6df7c6869c-kqhtm\" (UID: \"72ab8598-6730-4763-8a86-4c5ba57c6b65\") " pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"
Oct 30 18:53:20.237020 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.236997    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/72ab8598-6730-4763-8a86-4c5ba57c6b65-tmp\") pod \"kube-auth-proxy-6df7c6869c-kqhtm\" (UID: \"72ab8598-6730-4763-8a86-4c5ba57c6b65\") " pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"
Oct 30 18:53:20.237280 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.237261    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/72ab8598-6730-4763-8a86-4c5ba57c6b65-tls-certs\") pod \"kube-auth-proxy-6df7c6869c-kqhtm\" (UID: \"72ab8598-6730-4763-8a86-4c5ba57c6b65\") " pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"
Oct 30 18:53:20.294198 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.294173    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf4t9\" (UniqueName: \"kubernetes.io/projected/72ab8598-6730-4763-8a86-4c5ba57c6b65-kube-api-access-nf4t9\") pod \"kube-auth-proxy-6df7c6869c-kqhtm\" (UID: \"72ab8598-6730-4763-8a86-4c5ba57c6b65\") " pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"
Oct 30 18:53:20.384998 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.384971    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"
Oct 30 18:53:20.389666 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.389640    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:20.568343 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.568257    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:20.737537 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.737505    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:20.763326 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.757892    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm"]
Oct 30 18:53:20.767511 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:53:20.767480    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ab8598_6730_4763_8a86_4c5ba57c6b65.slice/crio-4598cdc129e9ce98212991a0681c8241f683e9fe14749c3135d30abfe06e811c WatchSource:0}: Error finding container 4598cdc129e9ce98212991a0681c8241f683e9fe14749c3135d30abfe06e811c: Status 404 returned error can't find the container with id 4598cdc129e9ce98212991a0681c8241f683e9fe14749c3135d30abfe06e811c
Oct 30 18:53:20.976489 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:20.976399    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:21.159181 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:21.157857    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:21.356321 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:21.356289    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:21.533332 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:21.533295    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:21.612164 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:21.612082    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm" event={"ID":"72ab8598-6730-4763-8a86-4c5ba57c6b65","Type":"ContainerStarted","Data":"4598cdc129e9ce98212991a0681c8241f683e9fe14749c3135d30abfe06e811c"}
Oct 30 18:53:23.601002 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:23.600322    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-k6957"
Oct 30 18:53:23.827875 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:23.827844    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 18:53:23.832694 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:23.832668    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"
Oct 30 18:53:23.871990 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:23.871926    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98r6s\" (UniqueName: \"kubernetes.io/projected/d000c687-d9f1-4c3b-ae0c-f4e6267306b9-kube-api-access-98r6s\") pod \"limitador-limitador-7c9b87d574-gpgrq\" (UID: \"d000c687-d9f1-4c3b-ae0c-f4e6267306b9\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"
Oct 30 18:53:23.872132 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:23.871996    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d000c687-d9f1-4c3b-ae0c-f4e6267306b9-config-file\") pod \"limitador-limitador-7c9b87d574-gpgrq\" (UID: \"d000c687-d9f1-4c3b-ae0c-f4e6267306b9\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"
Oct 30 18:53:23.881674 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:23.881652    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Oct 30 18:53:23.881674 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:23.881666    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Oct 30 18:53:23.881861 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:23.881723    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ptp75\""
Oct 30 18:53:23.881861 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:23.881776    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Oct 30 18:53:23.892434 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:23.892413    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 18:53:23.972713 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:23.972686    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98r6s\" (UniqueName: \"kubernetes.io/projected/d000c687-d9f1-4c3b-ae0c-f4e6267306b9-kube-api-access-98r6s\") pod \"limitador-limitador-7c9b87d574-gpgrq\" (UID: \"d000c687-d9f1-4c3b-ae0c-f4e6267306b9\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"
Oct 30 18:53:23.972869 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:23.972730    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d000c687-d9f1-4c3b-ae0c-f4e6267306b9-config-file\") pod \"limitador-limitador-7c9b87d574-gpgrq\" (UID: \"d000c687-d9f1-4c3b-ae0c-f4e6267306b9\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"
Oct 30 18:53:23.973406 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:23.973387    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d000c687-d9f1-4c3b-ae0c-f4e6267306b9-config-file\") pod \"limitador-limitador-7c9b87d574-gpgrq\" (UID: \"d000c687-d9f1-4c3b-ae0c-f4e6267306b9\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"
Oct 30 18:53:24.030141 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:24.030109    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-98r6s\" (UniqueName: \"kubernetes.io/projected/d000c687-d9f1-4c3b-ae0c-f4e6267306b9-kube-api-access-98r6s\") pod \"limitador-limitador-7c9b87d574-gpgrq\" (UID: \"d000c687-d9f1-4c3b-ae0c-f4e6267306b9\") " pod="kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"
Oct 30 18:53:24.109823 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:24.109779    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 18:53:24.149754 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:24.149661    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"
Oct 30 18:53:24.525735 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:24.525706    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 18:53:24.534170 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:53:24.534146    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd000c687_d9f1_4c3b_ae0c_f4e6267306b9.slice/crio-6e6c6174a7135b90f19fec0fe67f00931a9602103d4c6302fc9cf0fb08e9b2da WatchSource:0}: Error finding container 6e6c6174a7135b90f19fec0fe67f00931a9602103d4c6302fc9cf0fb08e9b2da: Status 404 returned error can't find the container with id 6e6c6174a7135b90f19fec0fe67f00931a9602103d4c6302fc9cf0fb08e9b2da
Oct 30 18:53:24.630464 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:24.630433    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-gpgrq" event={"ID":"d000c687-d9f1-4c3b-ae0c-f4e6267306b9","Type":"ContainerStarted","Data":"6e6c6174a7135b90f19fec0fe67f00931a9602103d4c6302fc9cf0fb08e9b2da"}
Oct 30 18:53:24.631856 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:24.631833    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm" event={"ID":"72ab8598-6730-4763-8a86-4c5ba57c6b65","Type":"ContainerStarted","Data":"98d0df7196fe48ebbad4d5810f47ccbba6c0a86324dcab52fe1117f82eed7f74"}
Oct 30 18:53:24.741247 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:24.741203    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6df7c6869c-kqhtm" podStartSLOduration=1.535370772 podStartE2EDuration="4.741189625s" totalImagesPullingTime="3.205818853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:24.740349805 +0000 UTC m=+377.686517707" watchObservedRunningTime="2026-10-30 18:53:24.741189625 +0000 UTC m=+377.687357524"
Oct 30 18:53:26.566151 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:26.566117    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:26.796222 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:26.796184    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:27.622579 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:27.622537    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:28.650134 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:28.650103    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7c9b87d574-gpgrq" event={"ID":"d000c687-d9f1-4c3b-ae0c-f4e6267306b9","Type":"ContainerStarted","Data":"9ebf2c97d31527fa9a6f74eb073e602e964383b73897c2f1330fd11ca38d651b"}
Oct 30 18:53:28.650512 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:28.650229    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"
Oct 30 18:53:28.760423 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:28.760375    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7c9b87d574-gpgrq" podStartSLOduration=2.591788978 podStartE2EDuration="5.760361003s" totalImagesPullingTime="3.168572025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:28.759409562 +0000 UTC m=+381.705577464" watchObservedRunningTime="2026-10-30 18:53:28.760361003 +0000 UTC m=+381.706528897"
Oct 30 18:53:30.503810 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:30.503770    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:30.731022 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:30.730983    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:33.382739 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:53:33.382692    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:53:37.541818 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:37.541781    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:37.721312 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:37.721272    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:39.655489 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:39.655466    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"
Oct 30 18:53:39.994977 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:39.994896    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:40.173748 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:40.173716    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:44.725856 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:44.725820    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:44.965097 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:44.965062    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:49.318609 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:49.318572    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:49.491469 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:49.491435    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:49.669859 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:49.669745    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:50.152606 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:50.152564    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:50.387649 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:50.387615    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:50.566222 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:50.566189    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:50.743278 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:50.743243    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:50.945108 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:50.945006    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:51.153835 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:51.153789    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:51.505826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:51.505799    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:54.184125 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:54.184091    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:54.361049 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:54.360999    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:55.754884 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:55.754848    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:55.937853 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:55.937797    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:56.476051 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:56.476013    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:56.657273 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:56.657244    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:56.985923 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:56.985887    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:57.174592 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:57.174552    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:59.424963 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:59.424925    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:53:59.657439 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:53:59.657397    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:02.747993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:02.747956    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:02.929221 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:02.929169    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:03.388341 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:54:03.388306    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:54:14.105364 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:14.105327    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:15.019167 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:15.019130    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:17.688769 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:17.688736    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:17.918477 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:17.918433    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:18.108507 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:18.108474    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:22.306581 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:22.306533    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:22.539158 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:22.539124    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:25.382853 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.382822    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"]
Oct 30 18:54:25.385590 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.385573    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:25.472284 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.472257    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8e6d648f-ac01-4171-afbb-7db22296c447-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:25.472498 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.472291    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e6d648f-ac01-4171-afbb-7db22296c447-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:25.472498 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.472314    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzvwp\" (UniqueName: \"kubernetes.io/projected/8e6d648f-ac01-4171-afbb-7db22296c447-kube-api-access-gzvwp\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:25.472498 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.472418    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/8e6d648f-ac01-4171-afbb-7db22296c447-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:25.500356 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.500331    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\""
Oct 30 18:54:25.500478 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.500370    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Oct 30 18:54:25.500478 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.500386    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Oct 30 18:54:25.500478 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.500387    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\""
Oct 30 18:54:25.500478 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.500476    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-4428p\""
Oct 30 18:54:25.500695 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.500681    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\""
Oct 30 18:54:25.502572 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.502557    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"]
Oct 30 18:54:25.573181 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.573142    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8e6d648f-ac01-4171-afbb-7db22296c447-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:25.573328 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.573201    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e6d648f-ac01-4171-afbb-7db22296c447-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:25.573328 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.573228    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzvwp\" (UniqueName: \"kubernetes.io/projected/8e6d648f-ac01-4171-afbb-7db22296c447-kube-api-access-gzvwp\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:25.573328 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:54:25.573249    5704 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found
Oct 30 18:54:25.573328 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.573306    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/8e6d648f-ac01-4171-afbb-7db22296c447-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:25.573541 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:54:25.573332    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8e6d648f-ac01-4171-afbb-7db22296c447-tls-cert podName:8e6d648f-ac01-4171-afbb-7db22296c447 nodeName:}" failed. No retries permitted until 2026-10-30 18:54:26.073311651 +0000 UTC m=+439.019479532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/8e6d648f-ac01-4171-afbb-7db22296c447-tls-cert") pod "usage-logs-tenancy-proxy-9c5d49bcd-k469z" (UID: "8e6d648f-ac01-4171-afbb-7db22296c447") : secret "usage-logs-tenancy-proxy-tls" not found
Oct 30 18:54:25.573541 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:54:25.573347    5704 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found
Oct 30 18:54:25.573541 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:54:25.573406    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8e6d648f-ac01-4171-afbb-7db22296c447-proxy-source podName:8e6d648f-ac01-4171-afbb-7db22296c447 nodeName:}" failed. No retries permitted until 2026-10-30 18:54:26.073387957 +0000 UTC m=+439.019555839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/8e6d648f-ac01-4171-afbb-7db22296c447-proxy-source") pod "usage-logs-tenancy-proxy-9c5d49bcd-k469z" (UID: "8e6d648f-ac01-4171-afbb-7db22296c447") : configmap "usage-tenancy-proxy-source" not found
Oct 30 18:54:25.573977 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.573955    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e6d648f-ac01-4171-afbb-7db22296c447-service-ca\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:25.613599 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.613554    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:25.633686 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.633634    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzvwp\" (UniqueName: \"kubernetes.io/projected/8e6d648f-ac01-4171-afbb-7db22296c447-kube-api-access-gzvwp\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:25.789811 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.789774    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:25.967758 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:25.967679    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:26.077174 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:26.077149    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8e6d648f-ac01-4171-afbb-7db22296c447-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:26.077342 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:26.077211    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/8e6d648f-ac01-4171-afbb-7db22296c447-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:26.077931 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:26.077912    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/8e6d648f-ac01-4171-afbb-7db22296c447-proxy-source\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:26.079607 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:26.079576    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/8e6d648f-ac01-4171-afbb-7db22296c447-tls-cert\") pod \"usage-logs-tenancy-proxy-9c5d49bcd-k469z\" (UID: \"8e6d648f-ac01-4171-afbb-7db22296c447\") " pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:26.297309 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:26.297285    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:26.664826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:26.664759    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"]
Oct 30 18:54:26.674051 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:54:26.674004    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e6d648f_ac01_4171_afbb_7db22296c447.slice/crio-9656f488faa8527affa4d6a12ea9c419d91e3aa6e1cfc415d221741a47015594 WatchSource:0}: Error finding container 9656f488faa8527affa4d6a12ea9c419d91e3aa6e1cfc415d221741a47015594: Status 404 returned error can't find the container with id 9656f488faa8527affa4d6a12ea9c419d91e3aa6e1cfc415d221741a47015594
Oct 30 18:54:26.893953 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:26.893918    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z" event={"ID":"8e6d648f-ac01-4171-afbb-7db22296c447","Type":"ContainerStarted","Data":"9656f488faa8527affa4d6a12ea9c419d91e3aa6e1cfc415d221741a47015594"}
Oct 30 18:54:33.397770 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:54:33.397724    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:54:34.726767 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.725127    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"]
Oct 30 18:54:34.745094 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.745066    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.822108 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.822058    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"]
Oct 30 18:54:34.852725 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.852686    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/617868bc-e87c-48c3-b358-7210bf1f388c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.852910 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.852768    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617868bc-e87c-48c3-b358-7210bf1f388c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.852910 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.852810    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/617868bc-e87c-48c3-b358-7210bf1f388c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.853061 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.852922    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/617868bc-e87c-48c3-b358-7210bf1f388c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.853061 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.852952    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/617868bc-e87c-48c3-b358-7210bf1f388c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.853061 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.852979    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xzp4\" (UniqueName: \"kubernetes.io/projected/617868bc-e87c-48c3-b358-7210bf1f388c-kube-api-access-8xzp4\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.853246 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.853129    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/617868bc-e87c-48c3-b358-7210bf1f388c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.954421 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.954385    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/617868bc-e87c-48c3-b358-7210bf1f388c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.954586 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.954427    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/617868bc-e87c-48c3-b358-7210bf1f388c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.954586 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.954462    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xzp4\" (UniqueName: \"kubernetes.io/projected/617868bc-e87c-48c3-b358-7210bf1f388c-kube-api-access-8xzp4\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.954586 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.954503    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/617868bc-e87c-48c3-b358-7210bf1f388c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.954739 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.954658    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/617868bc-e87c-48c3-b358-7210bf1f388c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.954739 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.954730    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617868bc-e87c-48c3-b358-7210bf1f388c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.954842 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.954771    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/617868bc-e87c-48c3-b358-7210bf1f388c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.955407 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.955384    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/617868bc-e87c-48c3-b358-7210bf1f388c-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.956947 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.956928    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/617868bc-e87c-48c3-b358-7210bf1f388c-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.957269 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.957245    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/617868bc-e87c-48c3-b358-7210bf1f388c-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.957363 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.957286    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/617868bc-e87c-48c3-b358-7210bf1f388c-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:34.957404 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:34.957390    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/617868bc-e87c-48c3-b358-7210bf1f388c-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:35.012311 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:35.012282    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/617868bc-e87c-48c3-b358-7210bf1f388c-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:35.012477 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:35.012374    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xzp4\" (UniqueName: \"kubernetes.io/projected/617868bc-e87c-48c3-b358-7210bf1f388c-kube-api-access-8xzp4\") pod \"istiod-openshift-gateway-7ddd9cfdc8-2tk6r\" (UID: \"617868bc-e87c-48c3-b358-7210bf1f388c\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:35.062172 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:35.062131    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:35.495181 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:35.495098    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"]
Oct 30 18:54:35.505935 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:35.505893    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:54:35.506090 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:35.505966    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:54:35.938867 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:35.938835    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r" event={"ID":"617868bc-e87c-48c3-b358-7210bf1f388c","Type":"ContainerStarted","Data":"9984b4ba08c82daa7c34960420b86de800abc357fb69a7ae17bbb5d2657e7449"}
Oct 30 18:54:35.939322 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:35.938881    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r" event={"ID":"617868bc-e87c-48c3-b358-7210bf1f388c","Type":"ContainerStarted","Data":"4f6c6592844077e725396a168518602f0ba382411c9f6de67548c49510a7b594"}
Oct 30 18:54:35.939322 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:35.938986    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:36.144930 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:36.144857    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r" podStartSLOduration=2.14483629 podStartE2EDuration="2.14483629s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:36.09783646 +0000 UTC m=+449.044004364" watchObservedRunningTime="2026-10-30 18:54:36.14483629 +0000 UTC m=+449.091004196"
Oct 30 18:54:36.947347 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:36.947321    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2tk6r"
Oct 30 18:54:37.155456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.155418    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"]
Oct 30 18:54:37.155691 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.155669    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7" podUID="cb7af7e1-435e-4996-8474-4f0144d33721" containerName="discovery" containerID="cri-o://6e1820a6cae2cd560c74cc49638972fa7ca4261b834b63b5d81d1b9b1f77e68b" gracePeriod=30
Oct 30 18:54:37.250524 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.250489    5704 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-884p7 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.21:8080/ready\": dial tcp 10.134.0.21:8080: connect: connection refused" start-of-body=
Oct 30 18:54:37.250697 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.250554    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7" podUID="cb7af7e1-435e-4996-8474-4f0144d33721" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.21:8080/ready\": dial tcp 10.134.0.21:8080: connect: connection refused"
Oct 30 18:54:37.522766 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.522736    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:54:37.578903 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.578872    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-istio-token\" (UniqueName: \"kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-istio-token\") pod \"cb7af7e1-435e-4996-8474-4f0144d33721\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") "
Oct 30 18:54:37.579092 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.578971    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-ca-configmap\") pod \"cb7af7e1-435e-4996-8474-4f0144d33721\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") "
Oct 30 18:54:37.579092 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.579025    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cb7af7e1-435e-4996-8474-4f0144d33721-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb7af7e1-435e-4996-8474-4f0144d33721-local-certs\") pod \"cb7af7e1-435e-4996-8474-4f0144d33721\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") "
Oct 30 18:54:37.579225 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.579109    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-dns-cert\") pod \"cb7af7e1-435e-4996-8474-4f0144d33721\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") "
Oct 30 18:54:37.579225 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.579145    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-kubeconfig\") pod \"cb7af7e1-435e-4996-8474-4f0144d33721\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") "
Oct 30 18:54:37.579225 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.579181    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-cacerts\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-cacerts\") pod \"cb7af7e1-435e-4996-8474-4f0144d33721\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") "
Oct 30 18:54:37.579225 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.579208    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-kube-api-access-hwrr4\" (UniqueName: \"kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-kube-api-access-hwrr4\") pod \"cb7af7e1-435e-4996-8474-4f0144d33721\" (UID: \"cb7af7e1-435e-4996-8474-4f0144d33721\") "
Oct 30 18:54:37.579414 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.579351    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-ca-configmap" pod "cb7af7e1-435e-4996-8474-4f0144d33721" (UID: "cb7af7e1-435e-4996-8474-4f0144d33721"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 18:54:37.579522 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.579506    5704 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-ca-configmap\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:37.581941 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.581913    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-dns-cert" pod "cb7af7e1-435e-4996-8474-4f0144d33721" (UID: "cb7af7e1-435e-4996-8474-4f0144d33721"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:37.582175 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.582055    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-kubeconfig" pod "cb7af7e1-435e-4996-8474-4f0144d33721" (UID: "cb7af7e1-435e-4996-8474-4f0144d33721"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:37.582175 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.582144    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb7af7e1-435e-4996-8474-4f0144d33721-local-certs" pod "cb7af7e1-435e-4996-8474-4f0144d33721" (UID: "cb7af7e1-435e-4996-8474-4f0144d33721"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 18:54:37.582704 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.582679    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-kube-api-access-hwrr4" pod "cb7af7e1-435e-4996-8474-4f0144d33721" (UID: "cb7af7e1-435e-4996-8474-4f0144d33721"). InnerVolumeSpecName "kube-api-access-hwrr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:37.582893 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.582864    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-cacerts" pod "cb7af7e1-435e-4996-8474-4f0144d33721" (UID: "cb7af7e1-435e-4996-8474-4f0144d33721"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 18:54:37.583011 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.582909    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-istio-token" pod "cb7af7e1-435e-4996-8474-4f0144d33721" (UID: "cb7af7e1-435e-4996-8474-4f0144d33721"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 18:54:37.680375 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.680330    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:37.680375 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.680375    5704 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cb7af7e1-435e-4996-8474-4f0144d33721-local-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:37.680628 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.680391    5704 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-csr-dns-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:37.680628 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.680402    5704 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-istio-kubeconfig\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:37.680628 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.680415    5704 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cb7af7e1-435e-4996-8474-4f0144d33721-cacerts\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:37.680628 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.680424    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwrr4\" (UniqueName: \"kubernetes.io/projected/cb7af7e1-435e-4996-8474-4f0144d33721-kube-api-access-hwrr4\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 18:54:37.952566 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.952471    5704 generic.go:356] "Generic (PLEG): container finished" podID="cb7af7e1-435e-4996-8474-4f0144d33721" containerID="6e1820a6cae2cd560c74cc49638972fa7ca4261b834b63b5d81d1b9b1f77e68b" exitCode=0
Oct 30 18:54:37.952566 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.952525    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"
Oct 30 18:54:37.953131 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.952564    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7" event={"ID":"cb7af7e1-435e-4996-8474-4f0144d33721","Type":"ContainerDied","Data":"6e1820a6cae2cd560c74cc49638972fa7ca4261b834b63b5d81d1b9b1f77e68b"}
Oct 30 18:54:37.953131 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.952619    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7" event={"ID":"cb7af7e1-435e-4996-8474-4f0144d33721","Type":"ContainerDied","Data":"0ce3937401e680f9b57130f4e0615fade2b82f144eff7e84d331023dc1d00a1b"}
Oct 30 18:54:37.953131 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.952645    5704 scope.go:122] "RemoveContainer" containerID="6e1820a6cae2cd560c74cc49638972fa7ca4261b834b63b5d81d1b9b1f77e68b"
Oct 30 18:54:37.962861 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.962839    5704 scope.go:122] "RemoveContainer" containerID="6e1820a6cae2cd560c74cc49638972fa7ca4261b834b63b5d81d1b9b1f77e68b"
Oct 30 18:54:37.963191 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:54:37.963163    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e1820a6cae2cd560c74cc49638972fa7ca4261b834b63b5d81d1b9b1f77e68b\": container with ID starting with 6e1820a6cae2cd560c74cc49638972fa7ca4261b834b63b5d81d1b9b1f77e68b not found: ID does not exist" containerID="6e1820a6cae2cd560c74cc49638972fa7ca4261b834b63b5d81d1b9b1f77e68b"
Oct 30 18:54:37.963284 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:37.963202    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e1820a6cae2cd560c74cc49638972fa7ca4261b834b63b5d81d1b9b1f77e68b"} err="failed to get container status \"6e1820a6cae2cd560c74cc49638972fa7ca4261b834b63b5d81d1b9b1f77e68b\": rpc error: code = NotFound desc = could not find container \"6e1820a6cae2cd560c74cc49638972fa7ca4261b834b63b5d81d1b9b1f77e68b\": container with ID starting with 6e1820a6cae2cd560c74cc49638972fa7ca4261b834b63b5d81d1b9b1f77e68b not found: ID does not exist"
Oct 30 18:54:38.105918 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:38.105886    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"]
Oct 30 18:54:38.151941 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:38.151906    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-884p7"]
Oct 30 18:54:39.179320 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:39.179282    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cb7af7e1-435e-4996-8474-4f0144d33721" path="/var/lib/kubelet/pods/cb7af7e1-435e-4996-8474-4f0144d33721/volumes"
Oct 30 18:54:39.661489 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:39.661452    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:39.894057 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:39.894007    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:40.075064 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.075000    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:40.267429 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.267391    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:40.445248 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.445119    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:40.627514 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.627474    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:40.763098 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.763066    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-gmcx6"]
Oct 30 18:54:40.763680 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.763659    5704 cpu_manager.go:414] "containerMap: removing container" podUID="cb7af7e1-435e-4996-8474-4f0144d33721" containerName="discovery"
Oct 30 18:54:40.763732 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.763684    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cb7af7e1-435e-4996-8474-4f0144d33721" containerName="discovery"
Oct 30 18:54:40.763828 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.763813    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="cb7af7e1-435e-4996-8474-4f0144d33721" containerName="discovery"
Oct 30 18:54:40.776988 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.776966    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-gmcx6"
Oct 30 18:54:40.812678 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.812649    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24bhn\" (UniqueName: \"kubernetes.io/projected/fca519ef-8ea0-4fdf-a9a2-e682e6896739-kube-api-access-24bhn\") pod \"model-serving-api-75bd89fdfc-gmcx6\" (UID: \"fca519ef-8ea0-4fdf-a9a2-e682e6896739\") " pod="opendatahub/model-serving-api-75bd89fdfc-gmcx6"
Oct 30 18:54:40.812797 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.812765    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fca519ef-8ea0-4fdf-a9a2-e682e6896739-tls-certs\") pod \"model-serving-api-75bd89fdfc-gmcx6\" (UID: \"fca519ef-8ea0-4fdf-a9a2-e682e6896739\") " pod="opendatahub/model-serving-api-75bd89fdfc-gmcx6"
Oct 30 18:54:40.816061 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.816019    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:40.824694 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.824669    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-gmcx6"]
Oct 30 18:54:40.869792 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.869764    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Oct 30 18:54:40.871331 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.871306    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-mf2dh\""
Oct 30 18:54:40.913979 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.913947    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24bhn\" (UniqueName: \"kubernetes.io/projected/fca519ef-8ea0-4fdf-a9a2-e682e6896739-kube-api-access-24bhn\") pod \"model-serving-api-75bd89fdfc-gmcx6\" (UID: \"fca519ef-8ea0-4fdf-a9a2-e682e6896739\") " pod="opendatahub/model-serving-api-75bd89fdfc-gmcx6"
Oct 30 18:54:40.914134 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.914022    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fca519ef-8ea0-4fdf-a9a2-e682e6896739-tls-certs\") pod \"model-serving-api-75bd89fdfc-gmcx6\" (UID: \"fca519ef-8ea0-4fdf-a9a2-e682e6896739\") " pod="opendatahub/model-serving-api-75bd89fdfc-gmcx6"
Oct 30 18:54:40.916755 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.916735    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fca519ef-8ea0-4fdf-a9a2-e682e6896739-tls-certs\") pod \"model-serving-api-75bd89fdfc-gmcx6\" (UID: \"fca519ef-8ea0-4fdf-a9a2-e682e6896739\") " pod="opendatahub/model-serving-api-75bd89fdfc-gmcx6"
Oct 30 18:54:40.972224 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:40.972197    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24bhn\" (UniqueName: \"kubernetes.io/projected/fca519ef-8ea0-4fdf-a9a2-e682e6896739-kube-api-access-24bhn\") pod \"model-serving-api-75bd89fdfc-gmcx6\" (UID: \"fca519ef-8ea0-4fdf-a9a2-e682e6896739\") " pod="opendatahub/model-serving-api-75bd89fdfc-gmcx6"
Oct 30 18:54:41.092722 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:41.092653    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-gmcx6"
Oct 30 18:54:41.473630 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:41.473608    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-gmcx6"]
Oct 30 18:54:41.481730 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:54:41.481704    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfca519ef_8ea0_4fdf_a9a2_e682e6896739.slice/crio-dee6c55833e628acda0cf32d5265a5fe17efec54a759a444facdd2194bba9b84 WatchSource:0}: Error finding container dee6c55833e628acda0cf32d5265a5fe17efec54a759a444facdd2194bba9b84: Status 404 returned error can't find the container with id dee6c55833e628acda0cf32d5265a5fe17efec54a759a444facdd2194bba9b84
Oct 30 18:54:41.974292 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:41.974260    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:41.974467 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:41.974334    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-gmcx6" event={"ID":"fca519ef-8ea0-4fdf-a9a2-e682e6896739","Type":"ContainerStarted","Data":"dee6c55833e628acda0cf32d5265a5fe17efec54a759a444facdd2194bba9b84"}
Oct 30 18:54:41.975867 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:41.975844    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z" event={"ID":"8e6d648f-ac01-4171-afbb-7db22296c447","Type":"ContainerStarted","Data":"30658481f636c91c2a10bff48990d92f83e276ea47ff4892db8e08c29b025fac"}
Oct 30 18:54:41.975995 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:41.975980    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:43.987741 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:43.987663    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-gmcx6" event={"ID":"fca519ef-8ea0-4fdf-a9a2-e682e6896739","Type":"ContainerStarted","Data":"321cc5d49dec011ca4fa01f7b08be51566799c8b4bf2fcdff9e7cade3b33c552"}
Oct 30 18:54:43.988101 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:43.987842    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-gmcx6"
Oct 30 18:54:44.097686 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:44.097638    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-gmcx6" podStartSLOduration=1.8493630840000002 podStartE2EDuration="4.097624603s" totalImagesPullingTime="2.248261519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:44.096758123 +0000 UTC m=+457.042926038" watchObservedRunningTime="2026-10-30 18:54:44.097624603 +0000 UTC m=+457.043792500"
Oct 30 18:54:44.098234 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:44.098201    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z" podStartSLOduration=4.333800163 podStartE2EDuration="19.098190165s" totalImagesPullingTime="14.764390002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:42.086224907 +0000 UTC m=+455.032392807" watchObservedRunningTime="2026-10-30 18:54:44.098190165 +0000 UTC m=+457.044358067"
Oct 30 18:54:47.407442 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:47.407408    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:47.594084 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:47.594051    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:47.792538 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:47.792496    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:47.993171 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:47.993107    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:48.175157 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:48.175089    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:48.352966 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:48.352924    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:48.553529 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:48.553498    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:48.731876 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:48.731853    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:52.988853 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:52.988827    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-k469z"
Oct 30 18:54:54.179210 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:54.179172    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:54.414406 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:54.414367    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:54.996183 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:54.996158    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-gmcx6"
Oct 30 18:54:56.295897 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:56.295857    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:56.483070 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:56.483012    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:56.662473 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:56.662398    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:54:56.858866 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:54:56.858830    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:02.705823 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:02.705789    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 18:55:03.381871 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:55:03.381839    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:55:07.458423 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:07.458351    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:07.693584 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:07.693552    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:07.934050 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:07.934008    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"]
Oct 30 18:55:08.163435 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.163394    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"]
Oct 30 18:55:08.163597 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.163495    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.212600 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.212546    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Oct 30 18:55:08.258301 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.258270    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-7w45v\""
Oct 30 18:55:08.258301 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.258271    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Oct 30 18:55:08.258301 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.258271    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Oct 30 18:55:08.258497 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.258291    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Oct 30 18:55:08.331311 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.331285    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19117a1-93c9-4a5a-a713-0f2a41bace8f-maas-api-tls\") pod \"maas-api-97d69cf74-lxpgv\" (UID: \"b19117a1-93c9-4a5a-a713-0f2a41bace8f\") " pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.331465 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.331329    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19117a1-93c9-4a5a-a713-0f2a41bace8f-metrics-certs\") pod \"maas-api-97d69cf74-lxpgv\" (UID: \"b19117a1-93c9-4a5a-a713-0f2a41bace8f\") " pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.331465 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.331347    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b19117a1-93c9-4a5a-a713-0f2a41bace8f-openshift-service-ca\") pod \"maas-api-97d69cf74-lxpgv\" (UID: \"b19117a1-93c9-4a5a-a713-0f2a41bace8f\") " pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.331465 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.331366    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hblrn\" (UniqueName: \"kubernetes.io/projected/b19117a1-93c9-4a5a-a713-0f2a41bace8f-kube-api-access-hblrn\") pod \"maas-api-97d69cf74-lxpgv\" (UID: \"b19117a1-93c9-4a5a-a713-0f2a41bace8f\") " pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.432612 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.432586    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19117a1-93c9-4a5a-a713-0f2a41bace8f-metrics-certs\") pod \"maas-api-97d69cf74-lxpgv\" (UID: \"b19117a1-93c9-4a5a-a713-0f2a41bace8f\") " pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.432746 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.432613    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b19117a1-93c9-4a5a-a713-0f2a41bace8f-openshift-service-ca\") pod \"maas-api-97d69cf74-lxpgv\" (UID: \"b19117a1-93c9-4a5a-a713-0f2a41bace8f\") " pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.432746 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.432632    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hblrn\" (UniqueName: \"kubernetes.io/projected/b19117a1-93c9-4a5a-a713-0f2a41bace8f-kube-api-access-hblrn\") pod \"maas-api-97d69cf74-lxpgv\" (UID: \"b19117a1-93c9-4a5a-a713-0f2a41bace8f\") " pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.432746 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.432686    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19117a1-93c9-4a5a-a713-0f2a41bace8f-maas-api-tls\") pod \"maas-api-97d69cf74-lxpgv\" (UID: \"b19117a1-93c9-4a5a-a713-0f2a41bace8f\") " pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.433305 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.433279    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b19117a1-93c9-4a5a-a713-0f2a41bace8f-openshift-service-ca\") pod \"maas-api-97d69cf74-lxpgv\" (UID: \"b19117a1-93c9-4a5a-a713-0f2a41bace8f\") " pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.434966 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.434946    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b19117a1-93c9-4a5a-a713-0f2a41bace8f-metrics-certs\") pod \"maas-api-97d69cf74-lxpgv\" (UID: \"b19117a1-93c9-4a5a-a713-0f2a41bace8f\") " pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.435122 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.435108    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b19117a1-93c9-4a5a-a713-0f2a41bace8f-maas-api-tls\") pod \"maas-api-97d69cf74-lxpgv\" (UID: \"b19117a1-93c9-4a5a-a713-0f2a41bace8f\") " pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.492637 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.492587    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hblrn\" (UniqueName: \"kubernetes.io/projected/b19117a1-93c9-4a5a-a713-0f2a41bace8f-kube-api-access-hblrn\") pod \"maas-api-97d69cf74-lxpgv\" (UID: \"b19117a1-93c9-4a5a-a713-0f2a41bace8f\") " pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.508457 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.508437    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:08.893527 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:08.893497    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"]
Oct 30 18:55:08.901666 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:55:08.901630    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb19117a1_93c9_4a5a_a713_0f2a41bace8f.slice/crio-f127ed6eb8c9db46ee2c2835fcfc206b0ba0fd4bd75006af93b4fdbe48778626 WatchSource:0}: Error finding container f127ed6eb8c9db46ee2c2835fcfc206b0ba0fd4bd75006af93b4fdbe48778626: Status 404 returned error can't find the container with id f127ed6eb8c9db46ee2c2835fcfc206b0ba0fd4bd75006af93b4fdbe48778626
Oct 30 18:55:09.091394 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:09.091366    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv" event={"ID":"b19117a1-93c9-4a5a-a713-0f2a41bace8f","Type":"ContainerStarted","Data":"f127ed6eb8c9db46ee2c2835fcfc206b0ba0fd4bd75006af93b4fdbe48778626"}
Oct 30 18:55:12.106656 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:12.106582    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv" event={"ID":"b19117a1-93c9-4a5a-a713-0f2a41bace8f","Type":"ContainerStarted","Data":"27ba0f1124180fcef4cc885cebfc5a7b7f36d4954eabdbdf908af7f8bf826340"}
Oct 30 18:55:12.107004 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:12.106715    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:12.218746 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:12.218695    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv" podStartSLOduration=2.320670652 podStartE2EDuration="5.218682545s" totalImagesPullingTime="2.898011893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:12.218007043 +0000 UTC m=+485.164174947" watchObservedRunningTime="2026-10-30 18:55:12.218682545 +0000 UTC m=+485.164850447"
Oct 30 18:55:18.115892 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:18.115862    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-97d69cf74-lxpgv"
Oct 30 18:55:33.382026 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:55:33.381980    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:55:56.089581 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:56.089540    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:55:57.406156 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:55:57.406123    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:03.381203 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:56:03.381103    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:56:27.044769 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:27.044693    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:27.232470 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:27.232435    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:33.381563 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:56:33.381527    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:56:36.426330 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:36.426285    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:36.658353 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:36.658267    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:38.841326 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:38.841288    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:39.019360 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:39.019320    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:44.766081 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:44.766050    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-5846b8599-xf797"]
Oct 30 18:56:44.768651 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:44.768635    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5846b8599-xf797"
Oct 30 18:56:44.807451 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:44.807421    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shjnj\" (UniqueName: \"kubernetes.io/projected/9e0abe2b-6b0d-4c71-9c95-bf49784088f0-kube-api-access-shjnj\") pod \"ai-gateway-controller-5846b8599-xf797\" (UID: \"9e0abe2b-6b0d-4c71-9c95-bf49784088f0\") " pod="opendatahub/ai-gateway-controller-5846b8599-xf797"
Oct 30 18:56:44.807581 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:44.807470    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e0abe2b-6b0d-4c71-9c95-bf49784088f0-metrics-certs\") pod \"ai-gateway-controller-5846b8599-xf797\" (UID: \"9e0abe2b-6b0d-4c71-9c95-bf49784088f0\") " pod="opendatahub/ai-gateway-controller-5846b8599-xf797"
Oct 30 18:56:44.863506 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:44.863477    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-metrics-tls\""
Oct 30 18:56:44.863506 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:44.863479    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-r7zq9\""
Oct 30 18:56:44.863662 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:44.863484    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Oct 30 18:56:44.864579 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:44.864562    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5846b8599-xf797"]
Oct 30 18:56:44.908362 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:44.908338    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shjnj\" (UniqueName: \"kubernetes.io/projected/9e0abe2b-6b0d-4c71-9c95-bf49784088f0-kube-api-access-shjnj\") pod \"ai-gateway-controller-5846b8599-xf797\" (UID: \"9e0abe2b-6b0d-4c71-9c95-bf49784088f0\") " pod="opendatahub/ai-gateway-controller-5846b8599-xf797"
Oct 30 18:56:44.908471 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:44.908381    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e0abe2b-6b0d-4c71-9c95-bf49784088f0-metrics-certs\") pod \"ai-gateway-controller-5846b8599-xf797\" (UID: \"9e0abe2b-6b0d-4c71-9c95-bf49784088f0\") " pod="opendatahub/ai-gateway-controller-5846b8599-xf797"
Oct 30 18:56:44.910791 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:44.910766    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9e0abe2b-6b0d-4c71-9c95-bf49784088f0-metrics-certs\") pod \"ai-gateway-controller-5846b8599-xf797\" (UID: \"9e0abe2b-6b0d-4c71-9c95-bf49784088f0\") " pod="opendatahub/ai-gateway-controller-5846b8599-xf797"
Oct 30 18:56:44.965700 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:44.965677    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shjnj\" (UniqueName: \"kubernetes.io/projected/9e0abe2b-6b0d-4c71-9c95-bf49784088f0-kube-api-access-shjnj\") pod \"ai-gateway-controller-5846b8599-xf797\" (UID: \"9e0abe2b-6b0d-4c71-9c95-bf49784088f0\") " pod="opendatahub/ai-gateway-controller-5846b8599-xf797"
Oct 30 18:56:45.081173 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:45.081117    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-5846b8599-xf797"
Oct 30 18:56:45.454661 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:45.454585    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-5846b8599-xf797"]
Oct 30 18:56:45.463091 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:56:45.463061    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e0abe2b_6b0d_4c71_9c95_bf49784088f0.slice/crio-ec919b91207711a14373e1934fa88db6c5a9e423ba25f920554ab31b608cbf65 WatchSource:0}: Error finding container ec919b91207711a14373e1934fa88db6c5a9e423ba25f920554ab31b608cbf65: Status 404 returned error can't find the container with id ec919b91207711a14373e1934fa88db6c5a9e423ba25f920554ab31b608cbf65
Oct 30 18:56:45.500053 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:45.500015    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5846b8599-xf797" event={"ID":"9e0abe2b-6b0d-4c71-9c95-bf49784088f0","Type":"ContainerStarted","Data":"ec919b91207711a14373e1934fa88db6c5a9e423ba25f920554ab31b608cbf65"}
Oct 30 18:56:48.514997 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:48.514958    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-5846b8599-xf797" event={"ID":"9e0abe2b-6b0d-4c71-9c95-bf49784088f0","Type":"ContainerStarted","Data":"11f4890d27f730fdd0bc0670fe0b5bcafb81808aaf670830621f177cfc395913"}
Oct 30 18:56:48.515417 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:48.515063    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-5846b8599-xf797"
Oct 30 18:56:48.628874 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:48.628815    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-5846b8599-xf797" podStartSLOduration=2.24708941 podStartE2EDuration="4.628803167s" totalImagesPullingTime="2.381713757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:48.628580859 +0000 UTC m=+581.574748772" watchObservedRunningTime="2026-10-30 18:56:48.628803167 +0000 UTC m=+581.574971069"
Oct 30 18:56:57.227180 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:57.227114    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:57.403966 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:57.403927    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:57.581665 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:57.581633    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:56:59.521072 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:56:59.521021    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-5846b8599-xf797"
Oct 30 18:57:03.381475 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:57:03.381444    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:57:09.021117 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:09.021083    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:09.199576 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:09.199542    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:10.702567 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:10.702526    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:11.556086 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:11.556049    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:11.783371 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:11.783340    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:12.111028 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:12.110995    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:13.456235 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:13.456204    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:13.682708 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:13.682671    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:20.473444 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:20.473410    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:20.757774 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:20.757737    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:23.974311 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:23.974277    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 18:57:27.811502 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.811466    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"]
Oct 30 18:57:27.815222 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.815199    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.868545 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.868511    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/244df3be-6afb-4788-a07e-533b1d3cc737-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.868687 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.868566    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/244df3be-6afb-4788-a07e-533b1d3cc737-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.868759 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.868693    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/244df3be-6afb-4788-a07e-533b1d3cc737-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.868759 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.868731    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4v27f\" (UniqueName: \"kubernetes.io/projected/244df3be-6afb-4788-a07e-533b1d3cc737-kube-api-access-4v27f\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.868870 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.868768    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/244df3be-6afb-4788-a07e-533b1d3cc737-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.868946 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.868920    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/244df3be-6afb-4788-a07e-533b1d3cc737-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.869075 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.869060    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Oct 30 18:57:27.914739 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.914710    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Oct 30 18:57:27.914999 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.914984    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-b4g6l\""
Oct 30 18:57:27.915092 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.915080    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Oct 30 18:57:27.916225 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.916207    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"]
Oct 30 18:57:27.962131 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.962101    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:27.969704 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.969684    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/244df3be-6afb-4788-a07e-533b1d3cc737-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.969810 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.969716    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/244df3be-6afb-4788-a07e-533b1d3cc737-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.969810 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.969736    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4v27f\" (UniqueName: \"kubernetes.io/projected/244df3be-6afb-4788-a07e-533b1d3cc737-kube-api-access-4v27f\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.969810 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.969770    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/244df3be-6afb-4788-a07e-533b1d3cc737-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.969968 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.969815    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/244df3be-6afb-4788-a07e-533b1d3cc737-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.969968 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.969846    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/244df3be-6afb-4788-a07e-533b1d3cc737-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.970184 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.970168    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/244df3be-6afb-4788-a07e-533b1d3cc737-model-cache\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.970252 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.970232    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/244df3be-6afb-4788-a07e-533b1d3cc737-home\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.970294 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.970255    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/244df3be-6afb-4788-a07e-533b1d3cc737-tmp-dir\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.972180 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.972164    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/244df3be-6afb-4788-a07e-533b1d3cc737-tls-certs\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:27.972312 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:27.972299    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/244df3be-6afb-4788-a07e-533b1d3cc737-dshm\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:28.027771 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:28.027748    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4v27f\" (UniqueName: \"kubernetes.io/projected/244df3be-6afb-4788-a07e-533b1d3cc737-kube-api-access-4v27f\") pod \"e2e-distinct-simulated-kserve-7b998b79d9-fvvkh\" (UID: \"244df3be-6afb-4788-a07e-533b1d3cc737\") " pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:28.127834 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:28.127781    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:28.501993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:28.501962    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"]
Oct 30 18:57:28.510937 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:57:28.510913    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod244df3be_6afb_4788_a07e_533b1d3cc737.slice/crio-2e9f198133d6b1035e4673bdfaabfb3330b2a75cbdf2116e9785c6a1941b7e58 WatchSource:0}: Error finding container 2e9f198133d6b1035e4673bdfaabfb3330b2a75cbdf2116e9785c6a1941b7e58: Status 404 returned error can't find the container with id 2e9f198133d6b1035e4673bdfaabfb3330b2a75cbdf2116e9785c6a1941b7e58
Oct 30 18:57:28.690292 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:28.690259    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh" event={"ID":"244df3be-6afb-4788-a07e-533b1d3cc737","Type":"ContainerStarted","Data":"2e9f198133d6b1035e4673bdfaabfb3330b2a75cbdf2116e9785c6a1941b7e58"}
Oct 30 18:57:30.659512 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:30.659477    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:30.701825 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:30.701777    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh" event={"ID":"244df3be-6afb-4788-a07e-533b1d3cc737","Type":"ContainerStarted","Data":"3152960d7a68e806040541a4f24b429a286e7a4cf84844611428153162b58d00"}
Oct 30 18:57:30.815263 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:30.815211    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh" podStartSLOduration=2.245635505 podStartE2EDuration="3.815197358s" totalImagesPullingTime="1.569561853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:30.814848284 +0000 UTC m=+623.761016188" watchObservedRunningTime="2026-10-30 18:57:30.815197358 +0000 UTC m=+623.761365302"
Oct 30 18:57:31.068541 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:31.068500    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:31.127931 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:31.127903    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:31.633053 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:31.632997    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:33.382144 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:57:33.382110    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:57:34.861938 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:34.861907    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"]
Oct 30 18:57:34.865663 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:34.865641    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:34.933852 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:34.933818    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7911229c-1ff9-4b92-bfcd-c060a9988f8e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:34.934156 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:34.934126    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7911229c-1ff9-4b92-bfcd-c060a9988f8e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:34.934280 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:34.934172    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7911229c-1ff9-4b92-bfcd-c060a9988f8e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:34.934280 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:34.934261    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7911229c-1ff9-4b92-bfcd-c060a9988f8e-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:34.934392 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:34.934314    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7911229c-1ff9-4b92-bfcd-c060a9988f8e-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:34.934453 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:34.934427    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62kcb\" (UniqueName: \"kubernetes.io/projected/7911229c-1ff9-4b92-bfcd-c060a9988f8e-kube-api-access-62kcb\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:34.960144 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:34.960119    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Oct 30 18:57:34.961416 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:34.961397    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"]
Oct 30 18:57:35.012242 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.012220    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:35.035206 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.035181    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7911229c-1ff9-4b92-bfcd-c060a9988f8e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:35.035324 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.035228    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7911229c-1ff9-4b92-bfcd-c060a9988f8e-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:35.035396 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.035378    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7911229c-1ff9-4b92-bfcd-c060a9988f8e-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:35.035453 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.035429    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62kcb\" (UniqueName: \"kubernetes.io/projected/7911229c-1ff9-4b92-bfcd-c060a9988f8e-kube-api-access-62kcb\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:35.035512 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.035480    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7911229c-1ff9-4b92-bfcd-c060a9988f8e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:35.035568 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.035516    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7911229c-1ff9-4b92-bfcd-c060a9988f8e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:35.035798 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.035777    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7911229c-1ff9-4b92-bfcd-c060a9988f8e-home\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:35.035952 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.035852    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7911229c-1ff9-4b92-bfcd-c060a9988f8e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:35.036083 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.035977    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7911229c-1ff9-4b92-bfcd-c060a9988f8e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:35.037496 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.037476    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7911229c-1ff9-4b92-bfcd-c060a9988f8e-dshm\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:35.037781 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.037761    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7911229c-1ff9-4b92-bfcd-c060a9988f8e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:35.092832 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.092812    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62kcb\" (UniqueName: \"kubernetes.io/projected/7911229c-1ff9-4b92-bfcd-c060a9988f8e-kube-api-access-62kcb\") pod \"e2e-trlp-test-simulated-kserve-69b588897d-dc9m6\" (UID: \"7911229c-1ff9-4b92-bfcd-c060a9988f8e\") " pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:35.177826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.177770    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:35.391306 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.391276    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 18:57:35.551663 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.551626    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"]
Oct 30 18:57:35.559566 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:57:35.559539    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7911229c_1ff9_4b92_bfcd_c060a9988f8e.slice/crio-5ee928e2d03fdae74d8b1332bb1f753c01d26eefe5412099978105d22f7095b7 WatchSource:0}: Error finding container 5ee928e2d03fdae74d8b1332bb1f753c01d26eefe5412099978105d22f7095b7: Status 404 returned error can't find the container with id 5ee928e2d03fdae74d8b1332bb1f753c01d26eefe5412099978105d22f7095b7
Oct 30 18:57:35.728936 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:35.728902    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6" event={"ID":"7911229c-1ff9-4b92-bfcd-c060a9988f8e","Type":"ContainerStarted","Data":"5ee928e2d03fdae74d8b1332bb1f753c01d26eefe5412099978105d22f7095b7"}
Oct 30 18:57:36.735042 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:36.735003    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6" event={"ID":"7911229c-1ff9-4b92-bfcd-c060a9988f8e","Type":"ContainerStarted","Data":"7a84d2c0aab8796dadb5cef2aff001d76888c5248bf04426b0dd73ffb9f3e73e"}
Oct 30 18:57:36.845704 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:36.845660    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6" podStartSLOduration=2.228773404 podStartE2EDuration="2.845646646s" totalImagesPullingTime="616.873242ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:36.84469911 +0000 UTC m=+629.790867024" watchObservedRunningTime="2026-10-30 18:57:36.845646646 +0000 UTC m=+629.791814548"
Oct 30 18:57:37.178134 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:37.178102    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:38.113701 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:38.113670    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:38.128262 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:38.128235    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:38.140334 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:38.140313    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:38.754529 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:38.754502    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-7b998b79d9-fvvkh"
Oct 30 18:57:41.699023 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.698993    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-g4zrw"]
Oct 30 18:57:41.702511 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.702491    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.751329 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.751291    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"echo-simulator-kserve-self-signed-certs\""
Oct 30 18:57:41.794582 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.794553    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.794706 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.794603    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.794706 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.794629    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kbtb\" (UniqueName: \"kubernetes.io/projected/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-kube-api-access-7kbtb\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.794800 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.794725    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-home\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.794800 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.794762    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.794800 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.794782    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.798237 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.798216    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-g4zrw"]
Oct 30 18:57:41.818577 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.818548    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:41.896197 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.896170    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.896343 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.896220    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.896343 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.896241    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kbtb\" (UniqueName: \"kubernetes.io/projected/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-kube-api-access-7kbtb\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.896343 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.896266    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-home\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.896343 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.896292    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.896602 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.896442    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.896660 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.896611    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-tmp-dir\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.896740 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.896717    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-model-cache\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.896841 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.896814    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-home\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.898414 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.898392    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-dshm\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.898824 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.898802    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-tls-certs\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:41.953886 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:41.953832    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kbtb\" (UniqueName: \"kubernetes.io/projected/1ada7c08-cef2-46d5-96b6-a6a8fddbc84a-kube-api-access-7kbtb\") pod \"echo-simulator-kserve-64b6f8f96c-g4zrw\" (UID: \"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a\") " pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:42.014469 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:42.014446    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:42.389855 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:42.389821    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/echo-simulator-kserve-64b6f8f96c-g4zrw"]
Oct 30 18:57:42.398250 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:57:42.398220    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ada7c08_cef2_46d5_96b6_a6a8fddbc84a.slice/crio-f34c164784364ade29bf9c5f28e1e0bd257ea4c9999002edf13d3fde13dec3e4 WatchSource:0}: Error finding container f34c164784364ade29bf9c5f28e1e0bd257ea4c9999002edf13d3fde13dec3e4: Status 404 returned error can't find the container with id f34c164784364ade29bf9c5f28e1e0bd257ea4c9999002edf13d3fde13dec3e4
Oct 30 18:57:42.762882 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:42.762852    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw" event={"ID":"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a","Type":"ContainerStarted","Data":"f34c164784364ade29bf9c5f28e1e0bd257ea4c9999002edf13d3fde13dec3e4"}
Oct 30 18:57:45.153906 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:45.153862    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:45.179773 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:45.179742    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:45.191453 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:45.191433    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:45.778392 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:45.778359    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw" event={"ID":"1ada7c08-cef2-46d5-96b6-a6a8fddbc84a","Type":"ContainerStarted","Data":"f3b3f88feb79985b3c6bd52adfdf84af9ad07d763ee6edb66b696322198a3c93"}
Oct 30 18:57:45.789530 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:45.789504    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-69b588897d-dc9m6"
Oct 30 18:57:45.895814 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:45.895722    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw" podStartSLOduration=2.319091744 podStartE2EDuration="4.895707869s" totalImagesPullingTime="2.576616125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:45.895278979 +0000 UTC m=+638.841446895" watchObservedRunningTime="2026-10-30 18:57:45.895707869 +0000 UTC m=+638.841875772"
Oct 30 18:57:46.015396 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:46.015369    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:48.205206 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.205131    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 18:57:48.756083 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.756051    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"]
Oct 30 18:57:48.759821 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.759798    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.808724 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.808701    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Oct 30 18:57:48.855517 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.855487    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"]
Oct 30 18:57:48.859126 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.859104    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.859216 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.859147    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lflbt\" (UniqueName: \"kubernetes.io/projected/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-kube-api-access-lflbt\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.859261 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.859218    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.859319 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.859263    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.859394 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.859375    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.859483 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.859467    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.890333 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.890305    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:48.960589 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.960557    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.960735 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.960609    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.960735 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.960643    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.960735 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.960668    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lflbt\" (UniqueName: \"kubernetes.io/projected/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-kube-api-access-lflbt\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.960735 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.960696    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.960735 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.960719    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.961022 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.960999    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.961111 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.961096    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.961152 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.961095    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-home\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.962820 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.962800    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-dshm\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:48.963086 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:48.963070    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:49.018633 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:49.018570    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lflbt\" (UniqueName: \"kubernetes.io/projected/fc21e17b-89c9-4a14-ad31-21d3ae548ee2-kube-api-access-lflbt\") pod \"premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8\" (UID: \"fc21e17b-89c9-4a14-ad31-21d3ae548ee2\") " pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:49.070619 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:49.070597    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:49.446546 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:49.446475    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"]
Oct 30 18:57:49.454179 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:57:49.454154    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc21e17b_89c9_4a14_ad31_21d3ae548ee2.slice/crio-8c553368304dfa6ef895b5ded6097ce22096001e53d15ebf4b0527ee0a3e6bdd WatchSource:0}: Error finding container 8c553368304dfa6ef895b5ded6097ce22096001e53d15ebf4b0527ee0a3e6bdd: Status 404 returned error can't find the container with id 8c553368304dfa6ef895b5ded6097ce22096001e53d15ebf4b0527ee0a3e6bdd
Oct 30 18:57:49.798026 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:49.797996    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8" event={"ID":"fc21e17b-89c9-4a14-ad31-21d3ae548ee2","Type":"ContainerStarted","Data":"8c553368304dfa6ef895b5ded6097ce22096001e53d15ebf4b0527ee0a3e6bdd"}
Oct 30 18:57:50.804050 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:50.804002    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8" event={"ID":"fc21e17b-89c9-4a14-ad31-21d3ae548ee2","Type":"ContainerStarted","Data":"57ccf53c57dc51638947b15f7ef744f89bde1d7cc8fb747da97b35dad2673aef"}
Oct 30 18:57:50.932634 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:50.932570    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8" podStartSLOduration=2.325834056 podStartE2EDuration="2.932556373s" totalImagesPullingTime="606.722317ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:50.930990265 +0000 UTC m=+643.877158167" watchObservedRunningTime="2026-10-30 18:57:50.932556373 +0000 UTC m=+643.878724276"
Oct 30 18:57:51.071686 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:51.071616    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:52.015462 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:52.015431    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:52.028700 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:52.028675    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:52.823429 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:52.823403    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/echo-simulator-kserve-64b6f8f96c-g4zrw"
Oct 30 18:57:56.616262 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:56.616232    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:57.580689 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:57.580654    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:57.810080 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:57.810022    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:57:59.071049 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:59.071004    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:59.083337 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:59.083315    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:57:59.854093 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:57:59.854067    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 18:58:00.609832 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:00.609798    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 18:58:03.382048 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:58:03.382000    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:58:11.387779 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.387748    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"]
Oct 30 18:58:11.393515 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.393499    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.442217 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.442194    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-4d7rj\""
Oct 30 18:58:11.459652 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.459629    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zmt2\" (UniqueName: \"kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-kube-api-access-5zmt2\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.459746 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.459665    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc15c72d-8800-49fb-b221-6490dd1dfb45-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.459746 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.459684    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.459826 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.459800    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.459868 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.459842    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.459868 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.459863    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.459967 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.459951    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.460009 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.459975    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.460009 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.460004    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.561258 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.561233    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.561391 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.561265    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.561391 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.561304    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.561391 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.561329    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.561391 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.561381    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.561574 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.561436    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zmt2\" (UniqueName: \"kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-kube-api-access-5zmt2\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.561574 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.561485    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc15c72d-8800-49fb-b221-6490dd1dfb45-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.561574 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.561520    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.561574 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.561564    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.561778 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.561683    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.561822 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.561778    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.562017 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.561988    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.562156 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.562097    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.562330 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.562309    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc15c72d-8800-49fb-b221-6490dd1dfb45-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.563725 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.563705    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.563814 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.563748    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.595451 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.595386    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"]
Oct 30 18:58:11.619482 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.619453    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.619622 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.619606    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zmt2\" (UniqueName: \"kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-kube-api-access-5zmt2\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-fd8nk\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:11.706195 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:11.706144    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:12.186121 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:12.186089    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"]
Oct 30 18:58:12.194143 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:58:12.194118    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc15c72d_8800_49fb_b221_6490dd1dfb45.slice/crio-4dff26fbf82dc62015a9b21437d45b8d8b47959c65124603de108937e8a5ec0d WatchSource:0}: Error finding container 4dff26fbf82dc62015a9b21437d45b8d8b47959c65124603de108937e8a5ec0d: Status 404 returned error can't find the container with id 4dff26fbf82dc62015a9b21437d45b8d8b47959c65124603de108937e8a5ec0d
Oct 30 18:58:12.901686 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:12.901652    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" event={"ID":"cc15c72d-8800-49fb-b221-6490dd1dfb45","Type":"ContainerStarted","Data":"4dff26fbf82dc62015a9b21437d45b8d8b47959c65124603de108937e8a5ec0d"}
Oct 30 18:58:16.629321 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:16.629283    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:58:16.629602 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:16.629362    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:58:16.629602 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:16.629397    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:58:16.921825 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:16.921751    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" event={"ID":"cc15c72d-8800-49fb-b221-6490dd1dfb45","Type":"ContainerStarted","Data":"c545e76bbf9bd9343fdcede8ae25e25d9b7ac1734f8a8e7e38125cbf3a7f10ac"}
Oct 30 18:58:17.115600 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:17.114693    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" podStartSLOduration=1.68243203 podStartE2EDuration="6.114673063s" totalImagesPullingTime="4.432241033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:17.081250667 +0000 UTC m=+670.027418569" watchObservedRunningTime="2026-10-30 18:58:17.114673063 +0000 UTC m=+670.060840966"
Oct 30 18:58:17.706819 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:17.706783    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:17.707975 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:17.707951    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:58:17.708088 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:17.708008    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 18:58:18.707307 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:18.707272    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:58:18.707775 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:18.707328    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 18:58:19.707445 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:19.707412    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:58:19.707843 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:19.707465    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 18:58:20.706864 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:20.706826    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:58:20.707062 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:20.706884    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 18:58:21.707112 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:21.707080    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:58:21.707516 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:21.707142    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 18:58:22.707512 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:22.707479    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:58:22.707918 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:22.707528    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 18:58:23.707187 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:23.707152    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused" start-of-body=
Oct 30 18:58:23.707434 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:23.707204    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": dial tcp 10.134.0.36:15021: connect: connection refused"
Oct 30 18:58:24.883119 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:24.883087    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:24.884081 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:24.884053    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:24.884773 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:24.884752    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 18:58:31.252484 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:31.252454    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:31.437729 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:31.437699    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:33.381798 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:58:33.381765    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:58:42.979661 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:42.979621    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:43.192214 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:43.192177    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:47.030385 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:47.030347    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:58:58.133359 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:58:58.133316    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:03.382138 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:59:03.382101    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:59:11.578047 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:11.577993    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:12.496185 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:12.496147    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:33.382515 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:59:33.382430    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 18:59:43.078324 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:43.078281    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:43.266494 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:43.266461    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:43.466993 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:43.466904    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:43.652452 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:43.652414    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:44.003864 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.003825    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:44.197673 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.197637    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:44.206109 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.206085    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"]
Oct 30 18:59:44.208947 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.208928    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.258096 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.258013    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b4e8ca3e-openshift-default-dockercfg-8d49f\""
Oct 30 18:59:44.289281 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.289247    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-token\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.289481 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.289288    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-envoy\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.289481 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.289372    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istiod-ca-cert\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.289481 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.289407    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-credential-socket\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.289481 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.289433    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-certs\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.289717 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.289525    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-data\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.289717 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.289566    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-podinfo\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.289717 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.289590    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-socket\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.289717 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.289648    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv6j6\" (UniqueName: \"kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-kube-api-access-xv6j6\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.305693 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.305662    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"]
Oct 30 18:59:44.384726 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.384700    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:44.390787 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.390768    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istiod-ca-cert\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.390882 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.390795    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-credential-socket\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.390882 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.390816    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-certs\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.390882 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.390842    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-data\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.391046 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.390896    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-podinfo\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.391101 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.391051    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-socket\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.391154 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.391139    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv6j6\" (UniqueName: \"kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-kube-api-access-xv6j6\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.391211 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.391179    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-token\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.391257 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.391222    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-envoy\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.391257 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.391237    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-credential-socket\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.391563 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.391541    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-data\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.391652 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.391563    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istiod-ca-cert\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.391652 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.391569    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-certs\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.391733 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.391670    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-socket\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.393357 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.393337    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-podinfo\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.393465 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.393449    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-envoy\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.448176 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.448157    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-token\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.448348 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.448330    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv6j6\" (UniqueName: \"kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-kube-api-access-xv6j6\") pod \"e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") " pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.520807 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.520747    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:44.563135 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.563096    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:44.738466 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.738424    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:44.919630 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.919556    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:44.949042 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.949007    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"]
Oct 30 18:59:44.958569 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:59:44.958542    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d79a7ee_5d5e_4fc5_bb91_8b3de8723222.slice/crio-48e0c6b667fbb1453cf57b99022b3601807f9bf3e2eb527b8b1b489c69457fde WatchSource:0}: Error finding container 48e0c6b667fbb1453cf57b99022b3601807f9bf3e2eb527b8b1b489c69457fde: Status 404 returned error can't find the container with id 48e0c6b667fbb1453cf57b99022b3601807f9bf3e2eb527b8b1b489c69457fde
Oct 30 18:59:44.960737 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.960706    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:44.960816 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.960803    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:44.960852 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:44.960835    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:45.338713 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:45.338677    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" event={"ID":"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222","Type":"ContainerStarted","Data":"eaba1cf1b308e1e7126cd9312e20dc8ab3f2c17cb94e4b450edf8faace33e4e2"}
Oct 30 18:59:45.339145 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:45.338730    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" event={"ID":"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222","Type":"ContainerStarted","Data":"48e0c6b667fbb1453cf57b99022b3601807f9bf3e2eb527b8b1b489c69457fde"}
Oct 30 18:59:45.452528 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:45.452478    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" podStartSLOduration=1.452465271 podStartE2EDuration="1.452465271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:45.452022536 +0000 UTC m=+758.398190440" watchObservedRunningTime="2026-10-30 18:59:45.452465271 +0000 UTC m=+758.398633174"
Oct 30 18:59:45.521709 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:45.521679    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:46.526221 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:46.526190    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:47.348500 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:47.348467    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:47.349271 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:47.349255    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 18:59:47.648484 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:47.648399    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:47.827667 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:47.827635    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:48.233718 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:48.233686    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:48.433255 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:48.433224    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:48.636899 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:48.636863    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:49.158776 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:49.158732    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:49.361240 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:49.361209    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:50.708827 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:50.708790    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:50.890129 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:50.890093    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:51.068377 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.068331    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:51.469793 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.469719    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:51.541071 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.541028    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"]
Oct 30 18:59:51.544309 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.544279    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.593170 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.593136    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ddb8b1dd-openshift-default-dockercfg-bxdsx\""
Oct 30 18:59:51.652344 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.652305    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"]
Oct 30 18:59:51.657205 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.657174    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-credential-socket\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.657380 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.657237    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-data\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.657380 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.657289    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-envoy\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.657380 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.657368    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-istio-token\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.657526 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.657399    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpxsd\" (UniqueName: \"kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-kube-api-access-qpxsd\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.657526 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.657442    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e868003e-829c-430f-8888-fb0f7d90c378-istiod-ca-cert\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.657526 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.657470    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e868003e-829c-430f-8888-fb0f7d90c378-istio-podinfo\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.657526 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.657498    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-certs\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.657708 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.657533    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-socket\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.703901 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.703868    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:51.758868 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.758842    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-istio-token\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.759336 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.758880    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpxsd\" (UniqueName: \"kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-kube-api-access-qpxsd\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.759336 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.758918    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e868003e-829c-430f-8888-fb0f7d90c378-istiod-ca-cert\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.759336 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.758945    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e868003e-829c-430f-8888-fb0f7d90c378-istio-podinfo\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.759336 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.758973    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-certs\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.759336 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.759078    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-socket\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.759336 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.759144    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-credential-socket\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.759336 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.759193    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-data\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.759336 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.759246    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-envoy\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.759685 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.759454    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-certs\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.759685 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.759488    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-socket\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.759685 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.759551    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-credential-socket\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.759834 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.759812    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e868003e-829c-430f-8888-fb0f7d90c378-istiod-ca-cert\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.759886 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.759820    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-data\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.761395 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.761376    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-envoy\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.761535 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.761518    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e868003e-829c-430f-8888-fb0f7d90c378-istio-podinfo\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.816155 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.816133    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-istio-token\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.816289 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.816271    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpxsd\" (UniqueName: \"kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-kube-api-access-qpxsd\") pod \"e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") " pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.858317 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.858299    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:51.889064 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:51.889010    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:52.018799 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.018731    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"]
Oct 30 18:59:52.021984 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.021942    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"
Oct 30 18:59:52.070751 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.070727    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-45e6849e\""
Oct 30 18:59:52.070751 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.070729    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-45e6849e-dockercfg-5nrxn\""
Oct 30 18:59:52.081156 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.081133    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"]
Oct 30 18:59:52.163981 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.163953    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqnj8\" (UniqueName: \"kubernetes.io/projected/6e6d3b75-1501-4978-8aec-835418f7bbcb-kube-api-access-nqnj8\") pod \"payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf\" (UID: \"6e6d3b75-1501-4978-8aec-835418f7bbcb\") " pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"
Oct 30 18:59:52.164135 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.163990    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e6d3b75-1501-4978-8aec-835418f7bbcb-plugins-config-volume\") pod \"payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf\" (UID: \"6e6d3b75-1501-4978-8aec-835418f7bbcb\") " pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"
Oct 30 18:59:52.265540 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.265504    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqnj8\" (UniqueName: \"kubernetes.io/projected/6e6d3b75-1501-4978-8aec-835418f7bbcb-kube-api-access-nqnj8\") pod \"payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf\" (UID: \"6e6d3b75-1501-4978-8aec-835418f7bbcb\") " pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"
Oct 30 18:59:52.265690 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.265550    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e6d3b75-1501-4978-8aec-835418f7bbcb-plugins-config-volume\") pod \"payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf\" (UID: \"6e6d3b75-1501-4978-8aec-835418f7bbcb\") " pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"
Oct 30 18:59:52.266414 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.266392    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e6d3b75-1501-4978-8aec-835418f7bbcb-plugins-config-volume\") pod \"payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf\" (UID: \"6e6d3b75-1501-4978-8aec-835418f7bbcb\") " pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"
Oct 30 18:59:52.300346 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.300290    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"]
Oct 30 18:59:52.308468 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:59:52.308445    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode868003e_829c_430f_8888_fb0f7d90c378.slice/crio-11dcf6f28460d75a97f37453f7b7fccf1d26edb4453dfda9e3314ee644818f68 WatchSource:0}: Error finding container 11dcf6f28460d75a97f37453f7b7fccf1d26edb4453dfda9e3314ee644818f68: Status 404 returned error can't find the container with id 11dcf6f28460d75a97f37453f7b7fccf1d26edb4453dfda9e3314ee644818f68
Oct 30 18:59:52.310619 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.310593    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:52.310684 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.310649    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:52.310684 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.310679    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:52.323286 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.323267    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqnj8\" (UniqueName: \"kubernetes.io/projected/6e6d3b75-1501-4978-8aec-835418f7bbcb-kube-api-access-nqnj8\") pod \"payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf\" (UID: \"6e6d3b75-1501-4978-8aec-835418f7bbcb\") " pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"
Oct 30 18:59:52.334413 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.334344    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"
Oct 30 18:59:52.373492 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.373460    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45" event={"ID":"e868003e-829c-430f-8888-fb0f7d90c378","Type":"ContainerStarted","Data":"302cccd1ae6381c400f3482a708bcf3cf6c736f82d811c158c28e5790a8b2f7b"}
Oct 30 18:59:52.373492 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.373515    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45" event={"ID":"e868003e-829c-430f-8888-fb0f7d90c378","Type":"ContainerStarted","Data":"11dcf6f28460d75a97f37453f7b7fccf1d26edb4453dfda9e3314ee644818f68"}
Oct 30 18:59:52.488396 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.488341    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45" podStartSLOduration=1.488326405 podStartE2EDuration="1.488326405s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:52.48731757 +0000 UTC m=+765.433485474" watchObservedRunningTime="2026-10-30 18:59:52.488326405 +0000 UTC m=+765.434494307"
Oct 30 18:59:52.707380 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.707305    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"]
Oct 30 18:59:52.715564 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:59:52.715537    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e6d3b75_1501_4978_8aec_835418f7bbcb.slice/crio-d2211e0710cfd43d1748ed95e65d7320440b38c0490501acd36145199ed0fe40 WatchSource:0}: Error finding container d2211e0710cfd43d1748ed95e65d7320440b38c0490501acd36145199ed0fe40: Status 404 returned error can't find the container with id d2211e0710cfd43d1748ed95e65d7320440b38c0490501acd36145199ed0fe40
Oct 30 18:59:52.717416 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.717400    5704 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 18:59:52.858912 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:52.858882    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:53.379349 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:53.379316    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf" event={"ID":"6e6d3b75-1501-4978-8aec-835418f7bbcb","Type":"ContainerStarted","Data":"d2211e0710cfd43d1748ed95e65d7320440b38c0490501acd36145199ed0fe40"}
Oct 30 18:59:53.863321 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:53.863294    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:53.929052 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:53.928988    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:54.155383 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:54.155305    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:54.386115 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:54.386082    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:54.387465 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:54.387433    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 18:59:55.067974 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.067923    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:55.571599 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.571564    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"]
Oct 30 18:59:55.574328 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.574309    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.624104 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.624079    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-df563ff7-openshift-default-dockercfg-lmj28\""
Oct 30 18:59:55.683456 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.681355    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"]
Oct 30 18:59:55.700635 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.700609    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-envoy\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.700776 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.700650    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-socket\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.700776 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.700741    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-credential-socket\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.700887 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.700778    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-certs\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.700930 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.700903    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-data\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.700968 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.700940    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istiod-ca-cert\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.701051 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.701018    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-token\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.701096 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.701065    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tgr7\" (UniqueName: \"kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-kube-api-access-4tgr7\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.701176 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.701158    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-podinfo\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.714944 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.714916    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:55.802720 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.802678    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-envoy\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.802913 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.802735    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-socket\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.802913 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.802786    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-credential-socket\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.802913 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.802825    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-certs\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.802913 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.802872    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-data\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.802913 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.802897    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istiod-ca-cert\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.803202 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.802929    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-token\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.803202 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.802958    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tgr7\" (UniqueName: \"kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-kube-api-access-4tgr7\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.803202 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.803015    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-podinfo\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.803658 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.803630    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-data\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.803914 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.803886    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-socket\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.804161 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.804138    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-credential-socket\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.804391 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.804367    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-certs\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.804952 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.804928    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istiod-ca-cert\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.806153 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.806127    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-podinfo\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.806812 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.806787    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-envoy\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.862540 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.862472    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-token\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.862540 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.862521    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tgr7\" (UniqueName: \"kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-kube-api-access-4tgr7\") pod \"e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") " pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.888364 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.888340    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:55.972691 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:55.971707    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:56.159108 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.159007    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:56.318756 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.318728    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"]
Oct 30 18:59:56.326950 ip-10-0-139-33 kubenswrapper[5704]: W1030 18:59:56.326917    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d1cb2c8_5d6f_40c0_8f7a_df170f7a2fcc.slice/crio-0bdf08240277ce308f23e49fcefeadbbd4fbe085c57ad4fb7aacce85a34d0118 WatchSource:0}: Error finding container 0bdf08240277ce308f23e49fcefeadbbd4fbe085c57ad4fb7aacce85a34d0118: Status 404 returned error can't find the container with id 0bdf08240277ce308f23e49fcefeadbbd4fbe085c57ad4fb7aacce85a34d0118
Oct 30 18:59:56.329206 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.329169    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:56.329281 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.329234    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:56.329281 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.329264    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 18:59:56.341705 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.341683    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:56.398679 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.398651    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k" event={"ID":"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc","Type":"ContainerStarted","Data":"0c8f87ba09d1fe11290e672cd424ca1029da552a93d1d8488856031505f5af2c"}
Oct 30 18:59:56.398798 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.398708    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k" event={"ID":"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc","Type":"ContainerStarted","Data":"0bdf08240277ce308f23e49fcefeadbbd4fbe085c57ad4fb7aacce85a34d0118"}
Oct 30 18:59:56.400340 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.400314    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf" event={"ID":"6e6d3b75-1501-4978-8aec-835418f7bbcb","Type":"ContainerStarted","Data":"3765632f27c839f67ee35f036e7608e9efd9d14e5532e447aa9255249c1a7a1d"}
Oct 30 18:59:56.400530 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.400510    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"
Oct 30 18:59:56.526833 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.523474    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k" podStartSLOduration=1.5233835199999999 podStartE2EDuration="1.52338352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:56.517416212 +0000 UTC m=+769.463584119" watchObservedRunningTime="2026-10-30 18:59:56.52338352 +0000 UTC m=+769.469551424"
Oct 30 18:59:56.548531 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.548497    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:56.653004 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.652951    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf" podStartSLOduration=2.768150757 podStartE2EDuration="5.652926238s" totalImagesPullingTime="2.884775481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:56.651908854 +0000 UTC m=+769.598076760" watchObservedRunningTime="2026-10-30 18:59:56.652926238 +0000 UTC m=+769.599094141"
Oct 30 18:59:56.749827 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.749768    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:56.888953 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:56.888863    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:57.893626 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:57.893599    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:58.409929 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:58.409897    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:58.410857 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:58.410836    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 18:59:58.508303 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:58.508270    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:58.692938 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:58.692847    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:59.100542 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.100503    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"]
Oct 30 18:59:59.104553 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.104529    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 18:59:59.155065 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.155009    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-45e6849e-serving-cert\""
Oct 30 18:59:59.202370 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.202340    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"]
Oct 30 18:59:59.246766 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.246738    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00706325-91b7-46ea-bdbf-9233227145d4-openshift-service-ca\") pod \"maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 18:59:59.246920 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.246832    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-maas-api-tls\") pod \"maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 18:59:59.246920 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.246884    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mvld\" (UniqueName: \"kubernetes.io/projected/00706325-91b7-46ea-bdbf-9233227145d4-kube-api-access-8mvld\") pod \"maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 18:59:59.247045 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.246961    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-metrics-certs\") pod \"maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 18:59:59.347513 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.347485    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mvld\" (UniqueName: \"kubernetes.io/projected/00706325-91b7-46ea-bdbf-9233227145d4-kube-api-access-8mvld\") pod \"maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 18:59:59.347640 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.347555    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-metrics-certs\") pod \"maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 18:59:59.347640 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.347614    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00706325-91b7-46ea-bdbf-9233227145d4-openshift-service-ca\") pod \"maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 18:59:59.347743 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.347685    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-maas-api-tls\") pod \"maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 18:59:59.347857 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:59:59.347842    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-serving-cert: secret "maas-api-e2e-migrate-45e6849e-serving-cert" not found
Oct 30 18:59:59.347916 ip-10-0-139-33 kubenswrapper[5704]: E1030 18:59:59.347906    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-maas-api-tls podName:00706325-91b7-46ea-bdbf-9233227145d4 nodeName:}" failed. No retries permitted until 2026-10-30 18:59:59.847891645 +0000 UTC m=+772.794059527 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-maas-api-tls") pod "maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g" (UID: "00706325-91b7-46ea-bdbf-9233227145d4") : secret "maas-api-e2e-migrate-45e6849e-serving-cert" not found
Oct 30 18:59:59.348346 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.348326    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00706325-91b7-46ea-bdbf-9233227145d4-openshift-service-ca\") pod \"maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 18:59:59.349950 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.349927    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-metrics-certs\") pod \"maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 18:59:59.353844 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.353793    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:59.405541 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.405510    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mvld\" (UniqueName: \"kubernetes.io/projected/00706325-91b7-46ea-bdbf-9233227145d4-kube-api-access-8mvld\") pod \"maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 18:59:59.558636 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.558600    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:59.735314 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.735237    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 18:59:59.853419 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.853391    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-maas-api-tls\") pod \"maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 18:59:59.855898 ip-10-0-139-33 kubenswrapper[5704]: I1030 18:59:59.855874    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-maas-api-tls\") pod \"maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 19:00:00.017660 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:00.017632    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 19:00:00.390764 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:00.390694    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"]
Oct 30 19:00:00.399108 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:00:00.399086    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00706325_91b7_46ea_bdbf_9233227145d4.slice/crio-3ff9033e02ac3c0823164b7b13755e331f134a4ff0e3ca9dd5ecfd21dfe54485 WatchSource:0}: Error finding container 3ff9033e02ac3c0823164b7b13755e331f134a4ff0e3ca9dd5ecfd21dfe54485: Status 404 returned error can't find the container with id 3ff9033e02ac3c0823164b7b13755e331f134a4ff0e3ca9dd5ecfd21dfe54485
Oct 30 19:00:00.420297 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:00.420270    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g" event={"ID":"00706325-91b7-46ea-bdbf-9233227145d4","Type":"ContainerStarted","Data":"3ff9033e02ac3c0823164b7b13755e331f134a4ff0e3ca9dd5ecfd21dfe54485"}
Oct 30 19:00:01.473633 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:01.473604    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:01.705670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:01.705583    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:01.839868 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:01.839836    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:00:02.431831 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:02.431797    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g" event={"ID":"00706325-91b7-46ea-bdbf-9233227145d4","Type":"ContainerStarted","Data":"25e19b39c5456e028989aea0f5ba01e1d3392ad811304c7c013837c5161c9fb6"}
Oct 30 19:00:02.431980 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:02.431919    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 19:00:02.543352 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:02.543302    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g" podStartSLOduration=2.531508068 podStartE2EDuration="3.543288673s" totalImagesPullingTime="1.011780605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:02.542125273 +0000 UTC m=+775.488293193" watchObservedRunningTime="2026-10-30 19:00:02.543288673 +0000 UTC m=+775.489456575"
Oct 30 19:00:03.383301 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:00:03.383269    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:00:04.247950 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:04.247917    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:04.433717 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:04.433685    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:04.615634 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:04.615536    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:05.431227 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:05.431179    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:05.657139 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:05.657104    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:05.809737 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:05.809705    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"]
Oct 30 19:00:05.813532 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:05.813513    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"
Oct 30 19:00:05.876020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:05.863612    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-ddb8b1dd\""
Oct 30 19:00:05.909787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:05.909759    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-ddb8b1dd-dockercfg-l5qpk\""
Oct 30 19:00:05.911776 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:05.911743    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"]
Oct 30 19:00:05.916294 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:05.916267    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl8xd\" (UniqueName: \"kubernetes.io/projected/887a2608-4fdd-485c-ae1d-fad8cc547ffb-kube-api-access-kl8xd\") pod \"payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2\" (UID: \"887a2608-4fdd-485c-ae1d-fad8cc547ffb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"
Oct 30 19:00:05.916420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:05.916349    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/887a2608-4fdd-485c-ae1d-fad8cc547ffb-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2\" (UID: \"887a2608-4fdd-485c-ae1d-fad8cc547ffb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"
Oct 30 19:00:06.017903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:06.017877    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/887a2608-4fdd-485c-ae1d-fad8cc547ffb-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2\" (UID: \"887a2608-4fdd-485c-ae1d-fad8cc547ffb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"
Oct 30 19:00:06.018067 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:06.017953    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl8xd\" (UniqueName: \"kubernetes.io/projected/887a2608-4fdd-485c-ae1d-fad8cc547ffb-kube-api-access-kl8xd\") pod \"payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2\" (UID: \"887a2608-4fdd-485c-ae1d-fad8cc547ffb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"
Oct 30 19:00:06.018579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:06.018559    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/887a2608-4fdd-485c-ae1d-fad8cc547ffb-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2\" (UID: \"887a2608-4fdd-485c-ae1d-fad8cc547ffb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"
Oct 30 19:00:06.075927 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:06.075860    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl8xd\" (UniqueName: \"kubernetes.io/projected/887a2608-4fdd-485c-ae1d-fad8cc547ffb-kube-api-access-kl8xd\") pod \"payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2\" (UID: \"887a2608-4fdd-485c-ae1d-fad8cc547ffb\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"
Oct 30 19:00:06.129562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:06.129537    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"
Oct 30 19:00:06.502118 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:06.502090    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"]
Oct 30 19:00:06.509971 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:00:06.509941    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod887a2608_4fdd_485c_ae1d_fad8cc547ffb.slice/crio-36ca7c83c7989d1557669d76ac3bed65d10cbd4a4bb29597246912137366cc19 WatchSource:0}: Error finding container 36ca7c83c7989d1557669d76ac3bed65d10cbd4a4bb29597246912137366cc19: Status 404 returned error can't find the container with id 36ca7c83c7989d1557669d76ac3bed65d10cbd4a4bb29597246912137366cc19
Oct 30 19:00:07.407627 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:07.407597    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"
Oct 30 19:00:07.463936 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:07.463907    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2" event={"ID":"887a2608-4fdd-485c-ae1d-fad8cc547ffb","Type":"ContainerStarted","Data":"f99a337b4fac01d8318d014a556134ee157c1a8317cf25ac2ae57b4d145ccf1d"}
Oct 30 19:00:07.464114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:07.463949    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2" event={"ID":"887a2608-4fdd-485c-ae1d-fad8cc547ffb","Type":"ContainerStarted","Data":"36ca7c83c7989d1557669d76ac3bed65d10cbd4a4bb29597246912137366cc19"}
Oct 30 19:00:07.464114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:07.463984    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"
Oct 30 19:00:07.626931 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:07.626879    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2" podStartSLOduration=2.6268651160000003 podStartE2EDuration="2.626865116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:07.625785859 +0000 UTC m=+780.571953761" watchObservedRunningTime="2026-10-30 19:00:07.626865116 +0000 UTC m=+780.573033018"
Oct 30 19:00:08.441648 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:08.441621    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 19:00:09.474360 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:09.474320    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:10.001534 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.001487    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:10.212231 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.212200    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:00:10.226550 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.226516    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:10.387937 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.387873    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"]
Oct 30 19:00:10.392775 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.392756    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"
Oct 30 19:00:10.442177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.442148    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-df563ff7\""
Oct 30 19:00:10.491366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.491330    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-df563ff7-dockercfg-cqlll\""
Oct 30 19:00:10.494455 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.494425    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"]
Oct 30 19:00:10.562017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.561988    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5spl\" (UniqueName: \"kubernetes.io/projected/a2820873-d0e0-4e1a-b227-f73f873ed679-kube-api-access-g5spl\") pod \"payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc\" (UID: \"a2820873-d0e0-4e1a-b227-f73f873ed679\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"
Oct 30 19:00:10.562185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.562116    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2820873-d0e0-4e1a-b227-f73f873ed679-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc\" (UID: \"a2820873-d0e0-4e1a-b227-f73f873ed679\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"
Oct 30 19:00:10.663421 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.663348    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2820873-d0e0-4e1a-b227-f73f873ed679-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc\" (UID: \"a2820873-d0e0-4e1a-b227-f73f873ed679\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"
Oct 30 19:00:10.663557 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.663467    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5spl\" (UniqueName: \"kubernetes.io/projected/a2820873-d0e0-4e1a-b227-f73f873ed679-kube-api-access-g5spl\") pod \"payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc\" (UID: \"a2820873-d0e0-4e1a-b227-f73f873ed679\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"
Oct 30 19:00:10.664198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.664179    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2820873-d0e0-4e1a-b227-f73f873ed679-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc\" (UID: \"a2820873-d0e0-4e1a-b227-f73f873ed679\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"
Oct 30 19:00:10.723649 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:10.723620    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5spl\" (UniqueName: \"kubernetes.io/projected/a2820873-d0e0-4e1a-b227-f73f873ed679-kube-api-access-g5spl\") pod \"payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc\" (UID: \"a2820873-d0e0-4e1a-b227-f73f873ed679\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"
Oct 30 19:00:11.005638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:11.005614    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"
Oct 30 19:00:11.377108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:11.376992    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"]
Oct 30 19:00:11.396002 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:00:11.395973    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2820873_d0e0_4e1a_b227_f73f873ed679.slice/crio-eb956fa1cd1f2671c92d52ea4d728f288e12e5282bcc4683c36356be88ac28d5 WatchSource:0}: Error finding container eb956fa1cd1f2671c92d52ea4d728f288e12e5282bcc4683c36356be88ac28d5: Status 404 returned error can't find the container with id eb956fa1cd1f2671c92d52ea4d728f288e12e5282bcc4683c36356be88ac28d5
Oct 30 19:00:11.493706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:11.493676    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc" event={"ID":"a2820873-d0e0-4e1a-b227-f73f873ed679","Type":"ContainerStarted","Data":"ea0a3dfe344121415bf5122ceee910287ed31748e3e3b90f22d0c6abdcc49dce"}
Oct 30 19:00:11.494084 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:11.493730    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc" event={"ID":"a2820873-d0e0-4e1a-b227-f73f873ed679","Type":"ContainerStarted","Data":"eb956fa1cd1f2671c92d52ea4d728f288e12e5282bcc4683c36356be88ac28d5"}
Oct 30 19:00:11.494084 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:11.493788    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"
Oct 30 19:00:11.606046 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:11.605993    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc" podStartSLOduration=1.605980203 podStartE2EDuration="1.605980203s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:11.605313276 +0000 UTC m=+784.551481191" watchObservedRunningTime="2026-10-30 19:00:11.605980203 +0000 UTC m=+784.552148106"
Oct 30 19:00:13.946725 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:13.946657    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"]
Oct 30 19:00:13.950139 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:13.950119    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:13.991761 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:13.991732    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-maas-api-tls\") pod \"maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:13.991894 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:13.991781    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2m55\" (UniqueName: \"kubernetes.io/projected/130e219c-85d5-4c72-90b1-a80e5e38396f-kube-api-access-w2m55\") pod \"maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:13.991894 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:13.991803    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/130e219c-85d5-4c72-90b1-a80e5e38396f-openshift-service-ca\") pod \"maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:13.991974 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:13.991947    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-metrics-certs\") pod \"maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:13.998606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:13.998584    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-ddb8b1dd-serving-cert\""
Oct 30 19:00:14.048152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.047670    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"]
Oct 30 19:00:14.093223 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.093195    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-metrics-certs\") pod \"maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:14.093404 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.093343    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-maas-api-tls\") pod \"maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:14.093404 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.093388    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2m55\" (UniqueName: \"kubernetes.io/projected/130e219c-85d5-4c72-90b1-a80e5e38396f-kube-api-access-w2m55\") pod \"maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:14.093543 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.093416    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/130e219c-85d5-4c72-90b1-a80e5e38396f-openshift-service-ca\") pod \"maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:14.093543 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:00:14.093525    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-serving-cert: secret "maas-api-e2e-derive-ddb8b1dd-serving-cert" not found
Oct 30 19:00:14.093658 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:00:14.093589    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-maas-api-tls podName:130e219c-85d5-4c72-90b1-a80e5e38396f nodeName:}" failed. No retries permitted until 2026-10-30 19:00:14.593566708 +0000 UTC m=+787.539734596 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-maas-api-tls") pod "maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll" (UID: "130e219c-85d5-4c72-90b1-a80e5e38396f") : secret "maas-api-e2e-derive-ddb8b1dd-serving-cert" not found
Oct 30 19:00:14.094337 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.094311    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/130e219c-85d5-4c72-90b1-a80e5e38396f-openshift-service-ca\") pod \"maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:14.096143 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.096122    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-metrics-certs\") pod \"maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:14.150993 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.150964    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2m55\" (UniqueName: \"kubernetes.io/projected/130e219c-85d5-4c72-90b1-a80e5e38396f-kube-api-access-w2m55\") pod \"maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:14.190083 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.190003    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:14.418513 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.418478    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:14.591168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.591133    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:14.602216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.602191    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-maas-api-tls\") pod \"maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:14.604456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.604432    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-maas-api-tls\") pod \"maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:14.862401 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:14.862342    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:15.245226 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:15.245145    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"]
Oct 30 19:00:15.248661 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:00:15.248637    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod130e219c_85d5_4c72_90b1_a80e5e38396f.slice/crio-aeb4e69f9d69edaf65f217aee753bcb32e3b89fb56194342376f3325a55ca652 WatchSource:0}: Error finding container aeb4e69f9d69edaf65f217aee753bcb32e3b89fb56194342376f3325a55ca652: Status 404 returned error can't find the container with id aeb4e69f9d69edaf65f217aee753bcb32e3b89fb56194342376f3325a55ca652
Oct 30 19:00:15.512676 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:15.512641    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll" event={"ID":"130e219c-85d5-4c72-90b1-a80e5e38396f","Type":"ContainerStarted","Data":"aeb4e69f9d69edaf65f217aee753bcb32e3b89fb56194342376f3325a55ca652"}
Oct 30 19:00:16.518883 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:16.518853    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll" event={"ID":"130e219c-85d5-4c72-90b1-a80e5e38396f","Type":"ContainerStarted","Data":"95c48128e6d66cbedbc6416144a023a38d3b95891b95714f4df64cb60304831b"}
Oct 30 19:00:16.519356 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:16.518977    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:16.638378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:16.638316    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll" podStartSLOduration=2.696631391 podStartE2EDuration="3.638297544s" totalImagesPullingTime="941.666153ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:16.636524801 +0000 UTC m=+789.582692705" watchObservedRunningTime="2026-10-30 19:00:16.638297544 +0000 UTC m=+789.584465448"
Oct 30 19:00:18.390224 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:18.390175    5704 patch_prober.go:29] interesting pod/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:00:18.390629 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:18.390251    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:00:18.471089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:18.471062    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"
Oct 30 19:00:19.236536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:19.236500    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:20.549314 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:20.549281    5704 generic.go:356] "Generic (PLEG): container finished" podID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerID="eaba1cf1b308e1e7126cd9312e20dc8ab3f2c17cb94e4b450edf8faace33e4e2" exitCode=0
Oct 30 19:00:20.549862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:20.549358    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" event={"ID":"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222","Type":"ContainerDied","Data":"eaba1cf1b308e1e7126cd9312e20dc8ab3f2c17cb94e4b450edf8faace33e4e2"}
Oct 30 19:00:20.549862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:20.549848    5704 scope.go:122] "RemoveContainer" containerID="eaba1cf1b308e1e7126cd9312e20dc8ab3f2c17cb94e4b450edf8faace33e4e2"
Oct 30 19:00:20.551173 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:20.551143    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:20.551292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:20.551227    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:20.551292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:20.551272    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:21.574915 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:21.574883    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" event={"ID":"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222","Type":"ContainerStarted","Data":"b1ee3f6f55cc0b496be99db42fe4de53f26cf0482695eec45ef0818575f0b1f8"}
Oct 30 19:00:22.501556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:22.501525    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"
Oct 30 19:00:22.531377 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:22.531347    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 19:00:22.544058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:22.541176    5704 patch_prober.go:29] interesting pod/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 19:00:22.544058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:22.541242    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 19:00:22.569059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:22.568779    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:00:23.525064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:23.523493    5704 patch_prober.go:29] interesting pod/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 19:00:23.525064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:23.523578    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 19:00:23.862774 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:23.862564    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:24.061052 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:24.060806    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:24.527058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:24.523396    5704 patch_prober.go:29] interesting pod/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 19:00:24.527058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:24.523483    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 19:00:25.523446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:25.523409    5704 patch_prober.go:29] interesting pod/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 19:00:25.523774 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:25.523733    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 19:00:26.530058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:26.521374    5704 patch_prober.go:29] interesting pod/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused" start-of-body=
Oct 30 19:00:26.530058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:26.521434    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.37:15021/healthz/ready\": dial tcp 10.134.0.37:15021: connect: connection refused"
Oct 30 19:00:27.525161 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:27.525132    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 19:00:27.525582 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:27.525558    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 19:00:27.526310 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:27.526292    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 19:00:28.941614 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:28.941589    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:00:29.097044 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:29.097012    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:29.618371 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:29.618342    5704 generic.go:356] "Generic (PLEG): container finished" podID="e868003e-829c-430f-8888-fb0f7d90c378" containerID="302cccd1ae6381c400f3482a708bcf3cf6c736f82d811c158c28e5790a8b2f7b" exitCode=0
Oct 30 19:00:29.618553 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:29.618419    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45" event={"ID":"e868003e-829c-430f-8888-fb0f7d90c378","Type":"ContainerDied","Data":"302cccd1ae6381c400f3482a708bcf3cf6c736f82d811c158c28e5790a8b2f7b"}
Oct 30 19:00:29.618836 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:29.618821    5704 scope.go:122] "RemoveContainer" containerID="302cccd1ae6381c400f3482a708bcf3cf6c736f82d811c158c28e5790a8b2f7b"
Oct 30 19:00:29.620077 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:29.620020    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:29.620179 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:29.620110    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:29.620179 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:29.620141    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:30.659365 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:30.659331    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45" event={"ID":"e868003e-829c-430f-8888-fb0f7d90c378","Type":"ContainerStarted","Data":"bbde0dba90ae568b5e3904137dd0d4afb54cf0d2748dc32b054f7be97fbc12cb"}
Oct 30 19:00:30.863058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:30.861438    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 19:00:30.863058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:30.861563    5704 patch_prober.go:29] interesting pod/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 19:00:30.863058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:30.861609    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45" podUID="e868003e-829c-430f-8888-fb0f7d90c378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 19:00:31.859765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:31.859332    5704 patch_prober.go:29] interesting pod/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 19:00:31.859765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:31.859393    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45" podUID="e868003e-829c-430f-8888-fb0f7d90c378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 19:00:32.860816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:32.860374    5704 patch_prober.go:29] interesting pod/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 19:00:32.860816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:32.860447    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45" podUID="e868003e-829c-430f-8888-fb0f7d90c378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 19:00:33.392241 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:00:33.392203    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:00:33.584120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:33.584085    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:33.869409 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:33.869373    5704 patch_prober.go:29] interesting pod/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 19:00:33.870423 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:33.870008    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45" podUID="e868003e-829c-430f-8888-fb0f7d90c378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 19:00:34.859463 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:34.859424    5704 patch_prober.go:29] interesting pod/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused" start-of-body=
Oct 30 19:00:34.859816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:34.859789    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45" podUID="e868003e-829c-430f-8888-fb0f7d90c378" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.38:15021/healthz/ready\": dial tcp 10.134.0.38:15021: connect: connection refused"
Oct 30 19:00:35.864159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:35.864131    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 19:00:35.864620 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:35.864521    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 19:00:35.865132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:35.865112    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 19:00:38.558756 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:38.558730    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:38.710965 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:38.710884    5704 generic.go:356] "Generic (PLEG): container finished" podID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerID="0c8f87ba09d1fe11290e672cd424ca1029da552a93d1d8488856031505f5af2c" exitCode=0
Oct 30 19:00:38.711106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:38.710958    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k" event={"ID":"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc","Type":"ContainerDied","Data":"0c8f87ba09d1fe11290e672cd424ca1029da552a93d1d8488856031505f5af2c"}
Oct 30 19:00:38.711335 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:38.711321    5704 scope.go:122] "RemoveContainer" containerID="0c8f87ba09d1fe11290e672cd424ca1029da552a93d1d8488856031505f5af2c"
Oct 30 19:00:38.712517 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:38.712492    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:38.712573 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:38.712558    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:38.712631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:38.712589    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:00:39.717828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:39.717791    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k" event={"ID":"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc","Type":"ContainerStarted","Data":"a9a705e4ee073d9494e4065a5c54a58de51d69f2db0a8def2f7d9c730ccd1b0d"}
Oct 30 19:00:39.889592 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:39.889557    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 19:00:39.890363 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:39.890339    5704 patch_prober.go:29] interesting pod/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 19:00:39.890504 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:39.890390    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k" podUID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 19:00:40.888713 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:40.888681    5704 patch_prober.go:29] interesting pod/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 19:00:40.889151 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:40.888745    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k" podUID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 19:00:41.889553 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:41.889523    5704 patch_prober.go:29] interesting pod/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 19:00:41.889993 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:41.889580    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k" podUID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 19:00:42.889433 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:42.889393    5704 patch_prober.go:29] interesting pod/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 19:00:42.889610 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:42.889447    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k" podUID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 19:00:43.410641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:43.410612    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 19:00:43.487655 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:43.487628    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:43.888897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:43.888856    5704 patch_prober.go:29] interesting pod/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body=
Oct 30 19:00:43.889073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:43.888928    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k" podUID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused"
Oct 30 19:00:44.893109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:44.893083    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 19:00:44.894077 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:44.894060    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 19:00:48.389087 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:48.388389    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:50.323311 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:50.323276    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:50.504017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:50.503981    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:52.333354 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:52.333318    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"]
Oct 30 19:00:52.718343 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:52.718260    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:52.945327 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:52.945293    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:53.211306 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:53.211260    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:56.226562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:56.226524    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:56.474007 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:56.473970    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:57.333824 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.333790    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf" podUID="6e6d3b75-1501-4978-8aec-835418f7bbcb" containerName="payload-processing" containerID="cri-o://3765632f27c839f67ee35f036e7608e9efd9d14e5532e447aa9255249c1a7a1d" gracePeriod=25
Oct 30 19:00:57.405789 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:00:57.405761    5704 logging.go:55] [core] [Channel #59 SubChannel #60]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.39:50052", ServerName: "10.134.0.39:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.39:50052: connect: connection refused"
Oct 30 19:00:57.535065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.532462    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"]
Oct 30 19:00:57.535065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.532700    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g" podUID="00706325-91b7-46ea-bdbf-9233227145d4" containerName="maas-api" containerID="cri-o://25e19b39c5456e028989aea0f5ba01e1d3392ad811304c7c013837c5161c9fb6" gracePeriod=30
Oct 30 19:00:57.673739 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.673710    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"
Oct 30 19:00:57.681362 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.681331    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:57.744632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.744591    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6e6d3b75-1501-4978-8aec-835418f7bbcb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e6d3b75-1501-4978-8aec-835418f7bbcb-plugins-config-volume\") pod \"6e6d3b75-1501-4978-8aec-835418f7bbcb\" (UID: \"6e6d3b75-1501-4978-8aec-835418f7bbcb\") "
Oct 30 19:00:57.744790 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.744694    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6e6d3b75-1501-4978-8aec-835418f7bbcb-kube-api-access-nqnj8\" (UniqueName: \"kubernetes.io/projected/6e6d3b75-1501-4978-8aec-835418f7bbcb-kube-api-access-nqnj8\") pod \"6e6d3b75-1501-4978-8aec-835418f7bbcb\" (UID: \"6e6d3b75-1501-4978-8aec-835418f7bbcb\") "
Oct 30 19:00:57.745877 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.745825    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e6d3b75-1501-4978-8aec-835418f7bbcb-plugins-config-volume" pod "6e6d3b75-1501-4978-8aec-835418f7bbcb" (UID: "6e6d3b75-1501-4978-8aec-835418f7bbcb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:57.760675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.760604    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e6d3b75-1501-4978-8aec-835418f7bbcb-kube-api-access-nqnj8" pod "6e6d3b75-1501-4978-8aec-835418f7bbcb" (UID: "6e6d3b75-1501-4978-8aec-835418f7bbcb"). InnerVolumeSpecName "kube-api-access-nqnj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:57.807535 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.807512    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 19:00:57.839410 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.839375    5704 generic.go:356] "Generic (PLEG): container finished" podID="6e6d3b75-1501-4978-8aec-835418f7bbcb" containerID="3765632f27c839f67ee35f036e7608e9efd9d14e5532e447aa9255249c1a7a1d" exitCode=0
Oct 30 19:00:57.839563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.839415    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"
Oct 30 19:00:57.839563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.839456    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf" event={"ID":"6e6d3b75-1501-4978-8aec-835418f7bbcb","Type":"ContainerDied","Data":"3765632f27c839f67ee35f036e7608e9efd9d14e5532e447aa9255249c1a7a1d"}
Oct 30 19:00:57.839563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.839512    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf" event={"ID":"6e6d3b75-1501-4978-8aec-835418f7bbcb","Type":"ContainerDied","Data":"d2211e0710cfd43d1748ed95e65d7320440b38c0490501acd36145199ed0fe40"}
Oct 30 19:00:57.839563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.839546    5704 scope.go:122] "RemoveContainer" containerID="3765632f27c839f67ee35f036e7608e9efd9d14e5532e447aa9255249c1a7a1d"
Oct 30 19:00:57.840761 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.840742    5704 generic.go:356] "Generic (PLEG): container finished" podID="00706325-91b7-46ea-bdbf-9233227145d4" containerID="25e19b39c5456e028989aea0f5ba01e1d3392ad811304c7c013837c5161c9fb6" exitCode=0
Oct 30 19:00:57.840861 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.840782    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"
Oct 30 19:00:57.840861 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.840816    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g" event={"ID":"00706325-91b7-46ea-bdbf-9233227145d4","Type":"ContainerDied","Data":"25e19b39c5456e028989aea0f5ba01e1d3392ad811304c7c013837c5161c9fb6"}
Oct 30 19:00:57.840965 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.840879    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g" event={"ID":"00706325-91b7-46ea-bdbf-9233227145d4","Type":"ContainerDied","Data":"3ff9033e02ac3c0823164b7b13755e331f134a4ff0e3ca9dd5ecfd21dfe54485"}
Oct 30 19:00:57.845788 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.845767    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6e6d3b75-1501-4978-8aec-835418f7bbcb-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:57.845884 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.845793    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqnj8\" (UniqueName: \"kubernetes.io/projected/6e6d3b75-1501-4978-8aec-835418f7bbcb-kube-api-access-nqnj8\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:57.850537 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.850523    5704 scope.go:122] "RemoveContainer" containerID="3765632f27c839f67ee35f036e7608e9efd9d14e5532e447aa9255249c1a7a1d"
Oct 30 19:00:57.850786 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:00:57.850771    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3765632f27c839f67ee35f036e7608e9efd9d14e5532e447aa9255249c1a7a1d\": container with ID starting with 3765632f27c839f67ee35f036e7608e9efd9d14e5532e447aa9255249c1a7a1d not found: ID does not exist" containerID="3765632f27c839f67ee35f036e7608e9efd9d14e5532e447aa9255249c1a7a1d"
Oct 30 19:00:57.850843 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.850801    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3765632f27c839f67ee35f036e7608e9efd9d14e5532e447aa9255249c1a7a1d"} err="failed to get container status \"3765632f27c839f67ee35f036e7608e9efd9d14e5532e447aa9255249c1a7a1d\": rpc error: code = NotFound desc = could not find container \"3765632f27c839f67ee35f036e7608e9efd9d14e5532e447aa9255249c1a7a1d\": container with ID starting with 3765632f27c839f67ee35f036e7608e9efd9d14e5532e447aa9255249c1a7a1d not found: ID does not exist"
Oct 30 19:00:57.850843 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.850817    5704 scope.go:122] "RemoveContainer" containerID="25e19b39c5456e028989aea0f5ba01e1d3392ad811304c7c013837c5161c9fb6"
Oct 30 19:00:57.858837 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.858817    5704 scope.go:122] "RemoveContainer" containerID="25e19b39c5456e028989aea0f5ba01e1d3392ad811304c7c013837c5161c9fb6"
Oct 30 19:00:57.859115 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:00:57.859095    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25e19b39c5456e028989aea0f5ba01e1d3392ad811304c7c013837c5161c9fb6\": container with ID starting with 25e19b39c5456e028989aea0f5ba01e1d3392ad811304c7c013837c5161c9fb6 not found: ID does not exist" containerID="25e19b39c5456e028989aea0f5ba01e1d3392ad811304c7c013837c5161c9fb6"
Oct 30 19:00:57.859167 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.859121    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25e19b39c5456e028989aea0f5ba01e1d3392ad811304c7c013837c5161c9fb6"} err="failed to get container status \"25e19b39c5456e028989aea0f5ba01e1d3392ad811304c7c013837c5161c9fb6\": rpc error: code = NotFound desc = could not find container \"25e19b39c5456e028989aea0f5ba01e1d3392ad811304c7c013837c5161c9fb6\": container with ID starting with 25e19b39c5456e028989aea0f5ba01e1d3392ad811304c7c013837c5161c9fb6 not found: ID does not exist"
Oct 30 19:00:57.946269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.946185    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-maas-api-tls\") pod \"00706325-91b7-46ea-bdbf-9233227145d4\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") "
Oct 30 19:00:57.946441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.946314    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00706325-91b7-46ea-bdbf-9233227145d4-kube-api-access-8mvld\" (UniqueName: \"kubernetes.io/projected/00706325-91b7-46ea-bdbf-9233227145d4-kube-api-access-8mvld\") pod \"00706325-91b7-46ea-bdbf-9233227145d4\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") "
Oct 30 19:00:57.946441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.946378    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/00706325-91b7-46ea-bdbf-9233227145d4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00706325-91b7-46ea-bdbf-9233227145d4-openshift-service-ca\") pod \"00706325-91b7-46ea-bdbf-9233227145d4\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") "
Oct 30 19:00:57.946441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.946404    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-metrics-certs\") pod \"00706325-91b7-46ea-bdbf-9233227145d4\" (UID: \"00706325-91b7-46ea-bdbf-9233227145d4\") "
Oct 30 19:00:57.946788 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.946760    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00706325-91b7-46ea-bdbf-9233227145d4-openshift-service-ca" pod "00706325-91b7-46ea-bdbf-9233227145d4" (UID: "00706325-91b7-46ea-bdbf-9233227145d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:00:57.948550 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.948526    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-metrics-certs" pod "00706325-91b7-46ea-bdbf-9233227145d4" (UID: "00706325-91b7-46ea-bdbf-9233227145d4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:00:57.948808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.948780    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-maas-api-tls" pod "00706325-91b7-46ea-bdbf-9233227145d4" (UID: "00706325-91b7-46ea-bdbf-9233227145d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:00:57.948808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:57.948801    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00706325-91b7-46ea-bdbf-9233227145d4-kube-api-access-8mvld" pod "00706325-91b7-46ea-bdbf-9233227145d4" (UID: "00706325-91b7-46ea-bdbf-9233227145d4"). InnerVolumeSpecName "kube-api-access-8mvld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:00:58.047484 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:58.047458    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:58.047484 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:58.047482    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mvld\" (UniqueName: \"kubernetes.io/projected/00706325-91b7-46ea-bdbf-9233227145d4-kube-api-access-8mvld\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:58.047484 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:58.047491    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/00706325-91b7-46ea-bdbf-9233227145d4-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:58.047687 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:58.047500    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/00706325-91b7-46ea-bdbf-9233227145d4-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:00:58.071385 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:58.071359    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"]
Oct 30 19:00:58.075516 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:58.075492    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf"]
Oct 30 19:00:58.253246 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:58.253221    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"]
Oct 30 19:00:58.261068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:58.261044    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-45e6849e-7b967c7b6f-qqr4g"]
Oct 30 19:00:58.405674 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:58.405646    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-migrate-45e6849e-549b477568-ksbbf" podUID="6e6d3b75-1501-4978-8aec-835418f7bbcb" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.39:50052\" within 1s: context deadline exceeded"
Oct 30 19:00:59.183265 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:59.183235    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00706325-91b7-46ea-bdbf-9233227145d4" path="/var/lib/kubelet/pods/00706325-91b7-46ea-bdbf-9233227145d4/volumes"
Oct 30 19:00:59.183836 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:59.183812    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6e6d3b75-1501-4978-8aec-835418f7bbcb" path="/var/lib/kubelet/pods/6e6d3b75-1501-4978-8aec-835418f7bbcb/volumes"
Oct 30 19:00:59.647196 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:59.647161    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:00:59.870157 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:00:59.870122    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:00.344904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:00.344857    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:00.626345 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:00.626259    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:00.819338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:00.819296    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:03.382546 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:03.382509    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:01:03.966045 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:03.966009    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"]
Oct 30 19:01:04.124073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:04.124025    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:01:04.246905 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:04.246867    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:04.478821 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:04.478787    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:04.738003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:04.737920    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"]
Oct 30 19:01:04.738183 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:04.738141    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll" podUID="130e219c-85d5-4c72-90b1-a80e5e38396f" containerName="maas-api" containerID="cri-o://95c48128e6d66cbedbc6416144a023a38d3b95891b95714f4df64cb60304831b" gracePeriod=30
Oct 30 19:01:04.863758 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:04.863731    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:04.880435 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:04.880407    5704 generic.go:356] "Generic (PLEG): container finished" podID="130e219c-85d5-4c72-90b1-a80e5e38396f" containerID="95c48128e6d66cbedbc6416144a023a38d3b95891b95714f4df64cb60304831b" exitCode=0
Oct 30 19:01:04.880575 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:04.880476    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll" event={"ID":"130e219c-85d5-4c72-90b1-a80e5e38396f","Type":"ContainerDied","Data":"95c48128e6d66cbedbc6416144a023a38d3b95891b95714f4df64cb60304831b"}
Oct 30 19:01:05.005405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.005382    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:01:05.053775 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.053738    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:05.120910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.120880    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-metrics-certs\") pod \"130e219c-85d5-4c72-90b1-a80e5e38396f\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") "
Oct 30 19:01:05.121126 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.121009    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/130e219c-85d5-4c72-90b1-a80e5e38396f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/130e219c-85d5-4c72-90b1-a80e5e38396f-openshift-service-ca\") pod \"130e219c-85d5-4c72-90b1-a80e5e38396f\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") "
Oct 30 19:01:05.121126 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.121061    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-maas-api-tls\") pod \"130e219c-85d5-4c72-90b1-a80e5e38396f\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") "
Oct 30 19:01:05.121126 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.121095    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/130e219c-85d5-4c72-90b1-a80e5e38396f-kube-api-access-w2m55\" (UniqueName: \"kubernetes.io/projected/130e219c-85d5-4c72-90b1-a80e5e38396f-kube-api-access-w2m55\") pod \"130e219c-85d5-4c72-90b1-a80e5e38396f\" (UID: \"130e219c-85d5-4c72-90b1-a80e5e38396f\") "
Oct 30 19:01:05.121391 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.121363    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/130e219c-85d5-4c72-90b1-a80e5e38396f-openshift-service-ca" pod "130e219c-85d5-4c72-90b1-a80e5e38396f" (UID: "130e219c-85d5-4c72-90b1-a80e5e38396f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:05.123443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.123414    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-metrics-certs" pod "130e219c-85d5-4c72-90b1-a80e5e38396f" (UID: "130e219c-85d5-4c72-90b1-a80e5e38396f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:01:05.123707 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.123680    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-maas-api-tls" pod "130e219c-85d5-4c72-90b1-a80e5e38396f" (UID: "130e219c-85d5-4c72-90b1-a80e5e38396f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:01:05.124334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.124305    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/130e219c-85d5-4c72-90b1-a80e5e38396f-kube-api-access-w2m55" pod "130e219c-85d5-4c72-90b1-a80e5e38396f" (UID: "130e219c-85d5-4c72-90b1-a80e5e38396f"). InnerVolumeSpecName "kube-api-access-w2m55". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:05.222409 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.222377    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/130e219c-85d5-4c72-90b1-a80e5e38396f-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:05.222409 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.222405    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:05.222576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.222419    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2m55\" (UniqueName: \"kubernetes.io/projected/130e219c-85d5-4c72-90b1-a80e5e38396f-kube-api-access-w2m55\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:05.222576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.222431    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/130e219c-85d5-4c72-90b1-a80e5e38396f-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:05.897755 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.897714    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"
Oct 30 19:01:05.897755 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.897739    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll" event={"ID":"130e219c-85d5-4c72-90b1-a80e5e38396f","Type":"ContainerDied","Data":"aeb4e69f9d69edaf65f217aee753bcb32e3b89fb56194342376f3325a55ca652"}
Oct 30 19:01:05.898358 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.897821    5704 scope.go:122] "RemoveContainer" containerID="95c48128e6d66cbedbc6416144a023a38d3b95891b95714f4df64cb60304831b"
Oct 30 19:01:05.965391 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:05.965365    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:06.050873 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:06.050839    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"]
Oct 30 19:01:06.096892 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:06.096853    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-ddb8b1dd-5459c447cd-fkpll"]
Oct 30 19:01:07.178588 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:07.178558    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="130e219c-85d5-4c72-90b1-a80e5e38396f" path="/var/lib/kubelet/pods/130e219c-85d5-4c72-90b1-a80e5e38396f/volumes"
Oct 30 19:01:07.410187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:07.410159    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:07.669224 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:07.669189    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:07.847694 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:07.847661    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:08.049567 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:08.049531    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:08.402632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:08.402555    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:08.658467 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:08.658363    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:08.839314 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:08.839281    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:08.963763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:08.963672    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"]
Oct 30 19:01:08.963963 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:08.963934    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45" podUID="e868003e-829c-430f-8888-fb0f7d90c378" containerName="istio-proxy" containerID="cri-o://bbde0dba90ae568b5e3904137dd0d4afb54cf0d2748dc32b054f7be97fbc12cb" gracePeriod=30
Oct 30 19:01:08.966521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:08.966466    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2" podUID="887a2608-4fdd-485c-ae1d-fad8cc547ffb" containerName="payload-pre-processing" containerID="cri-o://f99a337b4fac01d8318d014a556134ee157c1a8317cf25ac2ae57b4d145ccf1d" gracePeriod=25
Oct 30 19:01:09.025605 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.025571    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:09.212952 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.212926    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"
Oct 30 19:01:09.361619 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.361587    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/887a2608-4fdd-485c-ae1d-fad8cc547ffb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/887a2608-4fdd-485c-ae1d-fad8cc547ffb-plugins-config-volume\") pod \"887a2608-4fdd-485c-ae1d-fad8cc547ffb\" (UID: \"887a2608-4fdd-485c-ae1d-fad8cc547ffb\") "
Oct 30 19:01:09.361809 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.361668    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/887a2608-4fdd-485c-ae1d-fad8cc547ffb-kube-api-access-kl8xd\" (UniqueName: \"kubernetes.io/projected/887a2608-4fdd-485c-ae1d-fad8cc547ffb-kube-api-access-kl8xd\") pod \"887a2608-4fdd-485c-ae1d-fad8cc547ffb\" (UID: \"887a2608-4fdd-485c-ae1d-fad8cc547ffb\") "
Oct 30 19:01:09.361941 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.361918    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/887a2608-4fdd-485c-ae1d-fad8cc547ffb-plugins-config-volume" pod "887a2608-4fdd-485c-ae1d-fad8cc547ffb" (UID: "887a2608-4fdd-485c-ae1d-fad8cc547ffb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:09.362099 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.362084    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/887a2608-4fdd-485c-ae1d-fad8cc547ffb-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:09.363640 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.363618    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/887a2608-4fdd-485c-ae1d-fad8cc547ffb-kube-api-access-kl8xd" pod "887a2608-4fdd-485c-ae1d-fad8cc547ffb" (UID: "887a2608-4fdd-485c-ae1d-fad8cc547ffb"). InnerVolumeSpecName "kube-api-access-kl8xd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:09.433685 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.433651    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:09.463096 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.463073    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kl8xd\" (UniqueName: \"kubernetes.io/projected/887a2608-4fdd-485c-ae1d-fad8cc547ffb-kube-api-access-kl8xd\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:09.656454 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.656316    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:09.861258 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.861226    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:09.922025 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.921948    5704 generic.go:356] "Generic (PLEG): container finished" podID="887a2608-4fdd-485c-ae1d-fad8cc547ffb" containerID="f99a337b4fac01d8318d014a556134ee157c1a8317cf25ac2ae57b4d145ccf1d" exitCode=0
Oct 30 19:01:09.922025 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.922000    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"
Oct 30 19:01:09.922025 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.922049    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2" event={"ID":"887a2608-4fdd-485c-ae1d-fad8cc547ffb","Type":"ContainerDied","Data":"f99a337b4fac01d8318d014a556134ee157c1a8317cf25ac2ae57b4d145ccf1d"}
Oct 30 19:01:09.922274 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.922105    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2" event={"ID":"887a2608-4fdd-485c-ae1d-fad8cc547ffb","Type":"ContainerDied","Data":"36ca7c83c7989d1557669d76ac3bed65d10cbd4a4bb29597246912137366cc19"}
Oct 30 19:01:09.922274 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.922139    5704 scope.go:122] "RemoveContainer" containerID="f99a337b4fac01d8318d014a556134ee157c1a8317cf25ac2ae57b4d145ccf1d"
Oct 30 19:01:09.931598 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.931581    5704 scope.go:122] "RemoveContainer" containerID="f99a337b4fac01d8318d014a556134ee157c1a8317cf25ac2ae57b4d145ccf1d"
Oct 30 19:01:09.931872 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:09.931856    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f99a337b4fac01d8318d014a556134ee157c1a8317cf25ac2ae57b4d145ccf1d\": container with ID starting with f99a337b4fac01d8318d014a556134ee157c1a8317cf25ac2ae57b4d145ccf1d not found: ID does not exist" containerID="f99a337b4fac01d8318d014a556134ee157c1a8317cf25ac2ae57b4d145ccf1d"
Oct 30 19:01:09.931910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:09.931881    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f99a337b4fac01d8318d014a556134ee157c1a8317cf25ac2ae57b4d145ccf1d"} err="failed to get container status \"f99a337b4fac01d8318d014a556134ee157c1a8317cf25ac2ae57b4d145ccf1d\": rpc error: code = NotFound desc = could not find container \"f99a337b4fac01d8318d014a556134ee157c1a8317cf25ac2ae57b4d145ccf1d\": container with ID starting with f99a337b4fac01d8318d014a556134ee157c1a8317cf25ac2ae57b4d145ccf1d not found: ID does not exist"
Oct 30 19:01:10.074856 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.074816    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"]
Oct 30 19:01:10.077324 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.077298    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-ddb8b1dd-64457f5cc5-7x7t2"]
Oct 30 19:01:10.371118 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.371084    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"]
Oct 30 19:01:10.371681 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.371665    5704 cpu_manager.go:414] "containerMap: removing container" podUID="887a2608-4fdd-485c-ae1d-fad8cc547ffb" containerName="payload-pre-processing"
Oct 30 19:01:10.371735 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.371682    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="887a2608-4fdd-485c-ae1d-fad8cc547ffb" containerName="payload-pre-processing"
Oct 30 19:01:10.371735 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.371703    5704 cpu_manager.go:414] "containerMap: removing container" podUID="130e219c-85d5-4c72-90b1-a80e5e38396f" containerName="maas-api"
Oct 30 19:01:10.371735 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.371709    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="130e219c-85d5-4c72-90b1-a80e5e38396f" containerName="maas-api"
Oct 30 19:01:10.371735 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.371722    5704 cpu_manager.go:414] "containerMap: removing container" podUID="00706325-91b7-46ea-bdbf-9233227145d4" containerName="maas-api"
Oct 30 19:01:10.371735 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.371727    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00706325-91b7-46ea-bdbf-9233227145d4" containerName="maas-api"
Oct 30 19:01:10.371903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.371745    5704 cpu_manager.go:414] "containerMap: removing container" podUID="6e6d3b75-1501-4978-8aec-835418f7bbcb" containerName="payload-processing"
Oct 30 19:01:10.371903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.371752    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6e6d3b75-1501-4978-8aec-835418f7bbcb" containerName="payload-processing"
Oct 30 19:01:10.371903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.371822    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="130e219c-85d5-4c72-90b1-a80e5e38396f" containerName="maas-api"
Oct 30 19:01:10.371903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.371832    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="00706325-91b7-46ea-bdbf-9233227145d4" containerName="maas-api"
Oct 30 19:01:10.371903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.371841    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="6e6d3b75-1501-4978-8aec-835418f7bbcb" containerName="payload-processing"
Oct 30 19:01:10.371903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.371848    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="887a2608-4fdd-485c-ae1d-fad8cc547ffb" containerName="payload-pre-processing"
Oct 30 19:01:10.375129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.375110    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.424724 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.424687    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-9275c3-openshift-default-dockercfg-hq6vr\""
Oct 30 19:01:10.443963 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.443932    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"]
Oct 30 19:01:10.472708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.472679    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.472883 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.472727    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.472883 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.472807    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.472972 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.472949    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9vgk\" (UniqueName: \"kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-kube-api-access-w9vgk\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.473009 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.472979    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.473063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.473005    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.473111 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.473064    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.473153 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.473128    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.473186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.473154    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.503366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.503340    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:10.574477 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.574443    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9vgk\" (UniqueName: \"kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-kube-api-access-w9vgk\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.574477 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.574482    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.574723 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.574509    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.574723 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.574552    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.574723 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.574585    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.574723 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.574620    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.574723 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.574666    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.574723 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.574701    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.575048 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.574732    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.575048 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.574947    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.575048 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.575008    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.575212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.575173    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.575289 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.575270    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.575548 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.575527    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.576858 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.576842    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.577123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.577101    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.631880 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.631825    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.632072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.632049    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9vgk\" (UniqueName: \"kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-kube-api-access-w9vgk\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.686874 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.686846    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:10.708326 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:10.708297    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:11.112457 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:11.112426    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"]
Oct 30 19:01:11.120241 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:01:11.120215    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4aa4c2_580b_4c9a_a6bf_5e4de536f270.slice/crio-c3cb4e47a9779f1a10f28bfce02e050bcf8f9f1b4871ff028cccc4d26d7ed433 WatchSource:0}: Error finding container c3cb4e47a9779f1a10f28bfce02e050bcf8f9f1b4871ff028cccc4d26d7ed433: Status 404 returned error can't find the container with id c3cb4e47a9779f1a10f28bfce02e050bcf8f9f1b4871ff028cccc4d26d7ed433
Oct 30 19:01:11.122502 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:11.122473    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:11.122581 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:11.122530    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:11.122581 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:11.122560    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:11.180488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:11.180463    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="887a2608-4fdd-485c-ae1d-fad8cc547ffb" path="/var/lib/kubelet/pods/887a2608-4fdd-485c-ae1d-fad8cc547ffb/volumes"
Oct 30 19:01:11.696626 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:11.696592    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:11.875704 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:11.875654    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:11.933508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:11.933476    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j" event={"ID":"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270","Type":"ContainerStarted","Data":"ca6aa4e2f18fd15e4e12d15b68ae016b72b9919ffaa7d9580feda18e9491392b"}
Oct 30 19:01:11.933694 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:11.933536    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j" event={"ID":"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270","Type":"ContainerStarted","Data":"c3cb4e47a9779f1a10f28bfce02e050bcf8f9f1b4871ff028cccc4d26d7ed433"}
Oct 30 19:01:12.048630 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:12.048578    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j" podStartSLOduration=2.048561384 podStartE2EDuration="2.048561384s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:12.048274672 +0000 UTC m=+844.994442690" watchObservedRunningTime="2026-10-30 19:01:12.048561384 +0000 UTC m=+844.994729288"
Oct 30 19:01:12.381377 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:12.381288    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"]
Oct 30 19:01:12.381580 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:12.381552    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy" containerID="cri-o://b1ee3f6f55cc0b496be99db42fe4de53f26cf0482695eec45ef0818575f0b1f8" gracePeriod=30
Oct 30 19:01:12.454468 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:12.454438    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:12.528014 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:12.527985    5704 patch_prober.go:29] interesting pod/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:01:12.528179 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:12.528061    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:01:12.681681 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:12.681598    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:12.687424 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:12.687404    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:12.692655 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:12.692635    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:12.707440 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:12.707416    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"]
Oct 30 19:01:12.939083 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:12.938991    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:12.939894 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:12.939876    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:12.990862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:12.990837    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:13.175707 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:13.175676    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:13.715447 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:13.715419    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:01:14.227057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.227024    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 19:01:14.325093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325065    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-envoy\") pod \"e868003e-829c-430f-8888-fb0f7d90c378\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") "
Oct 30 19:01:14.325290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325113    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-istio-token\" (UniqueName: \"kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-istio-token\") pod \"e868003e-829c-430f-8888-fb0f7d90c378\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") "
Oct 30 19:01:14.325290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325136    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-kube-api-access-qpxsd\" (UniqueName: \"kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-kube-api-access-qpxsd\") pod \"e868003e-829c-430f-8888-fb0f7d90c378\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") "
Oct 30 19:01:14.325290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325164    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-data\") pod \"e868003e-829c-430f-8888-fb0f7d90c378\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") "
Oct 30 19:01:14.325290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325195    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-certs\") pod \"e868003e-829c-430f-8888-fb0f7d90c378\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") "
Oct 30 19:01:14.325290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325269    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e868003e-829c-430f-8888-fb0f7d90c378-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e868003e-829c-430f-8888-fb0f7d90c378-istio-podinfo\") pod \"e868003e-829c-430f-8888-fb0f7d90c378\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") "
Oct 30 19:01:14.325290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325291    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e868003e-829c-430f-8888-fb0f7d90c378-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e868003e-829c-430f-8888-fb0f7d90c378-istiod-ca-cert\") pod \"e868003e-829c-430f-8888-fb0f7d90c378\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") "
Oct 30 19:01:14.325618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325358    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-credential-socket\") pod \"e868003e-829c-430f-8888-fb0f7d90c378\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") "
Oct 30 19:01:14.325618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325410    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-socket\") pod \"e868003e-829c-430f-8888-fb0f7d90c378\" (UID: \"e868003e-829c-430f-8888-fb0f7d90c378\") "
Oct 30 19:01:14.325618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325457    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-data" pod "e868003e-829c-430f-8888-fb0f7d90c378" (UID: "e868003e-829c-430f-8888-fb0f7d90c378"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:14.325618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325542    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-certs" pod "e868003e-829c-430f-8888-fb0f7d90c378" (UID: "e868003e-829c-430f-8888-fb0f7d90c378"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:14.325812 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325732    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-socket" pod "e868003e-829c-430f-8888-fb0f7d90c378" (UID: "e868003e-829c-430f-8888-fb0f7d90c378"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:14.325812 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325768    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-credential-socket" pod "e868003e-829c-430f-8888-fb0f7d90c378" (UID: "e868003e-829c-430f-8888-fb0f7d90c378"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:14.325812 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325794    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e868003e-829c-430f-8888-fb0f7d90c378-istiod-ca-cert" pod "e868003e-829c-430f-8888-fb0f7d90c378" (UID: "e868003e-829c-430f-8888-fb0f7d90c378"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:14.326009 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.325989    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e868003e-829c-430f-8888-fb0f7d90c378-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:14.326103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.326014    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:14.326103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.326059    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:14.326103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.326080    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:14.326103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.326095    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:14.327443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.327414    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-envoy" pod "e868003e-829c-430f-8888-fb0f7d90c378" (UID: "e868003e-829c-430f-8888-fb0f7d90c378"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:14.327543 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.327485    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e868003e-829c-430f-8888-fb0f7d90c378-istio-podinfo" pod "e868003e-829c-430f-8888-fb0f7d90c378" (UID: "e868003e-829c-430f-8888-fb0f7d90c378"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:01:14.327743 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.327728    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-istio-token" pod "e868003e-829c-430f-8888-fb0f7d90c378" (UID: "e868003e-829c-430f-8888-fb0f7d90c378"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:14.327984 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.327971    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-kube-api-access-qpxsd" pod "e868003e-829c-430f-8888-fb0f7d90c378" (UID: "e868003e-829c-430f-8888-fb0f7d90c378"). InnerVolumeSpecName "kube-api-access-qpxsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:14.427252 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.427223    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e868003e-829c-430f-8888-fb0f7d90c378-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:14.427252 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.427246    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e868003e-829c-430f-8888-fb0f7d90c378-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:14.427252 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.427256    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:14.427445 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.427265    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpxsd\" (UniqueName: \"kubernetes.io/projected/e868003e-829c-430f-8888-fb0f7d90c378-kube-api-access-qpxsd\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:14.949055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.949004    5704 generic.go:356] "Generic (PLEG): container finished" podID="e868003e-829c-430f-8888-fb0f7d90c378" containerID="bbde0dba90ae568b5e3904137dd0d4afb54cf0d2748dc32b054f7be97fbc12cb" exitCode=0
Oct 30 19:01:14.949440 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.949086    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"
Oct 30 19:01:14.949440 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.949090    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45" event={"ID":"e868003e-829c-430f-8888-fb0f7d90c378","Type":"ContainerDied","Data":"bbde0dba90ae568b5e3904137dd0d4afb54cf0d2748dc32b054f7be97fbc12cb"}
Oct 30 19:01:14.949440 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.949149    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45" event={"ID":"e868003e-829c-430f-8888-fb0f7d90c378","Type":"ContainerDied","Data":"11dcf6f28460d75a97f37453f7b7fccf1d26edb4453dfda9e3314ee644818f68"}
Oct 30 19:01:14.949440 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.949172    5704 scope.go:122] "RemoveContainer" containerID="bbde0dba90ae568b5e3904137dd0d4afb54cf0d2748dc32b054f7be97fbc12cb"
Oct 30 19:01:14.958541 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.958510    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:14.958806 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.958789    5704 scope.go:122] "RemoveContainer" containerID="302cccd1ae6381c400f3482a708bcf3cf6c736f82d811c158c28e5790a8b2f7b"
Oct 30 19:01:14.966592 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.966573    5704 scope.go:122] "RemoveContainer" containerID="bbde0dba90ae568b5e3904137dd0d4afb54cf0d2748dc32b054f7be97fbc12cb"
Oct 30 19:01:14.966849 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:14.966832    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bbde0dba90ae568b5e3904137dd0d4afb54cf0d2748dc32b054f7be97fbc12cb\": container with ID starting with bbde0dba90ae568b5e3904137dd0d4afb54cf0d2748dc32b054f7be97fbc12cb not found: ID does not exist" containerID="bbde0dba90ae568b5e3904137dd0d4afb54cf0d2748dc32b054f7be97fbc12cb"
Oct 30 19:01:14.966897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.966857    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bbde0dba90ae568b5e3904137dd0d4afb54cf0d2748dc32b054f7be97fbc12cb"} err="failed to get container status \"bbde0dba90ae568b5e3904137dd0d4afb54cf0d2748dc32b054f7be97fbc12cb\": rpc error: code = NotFound desc = could not find container \"bbde0dba90ae568b5e3904137dd0d4afb54cf0d2748dc32b054f7be97fbc12cb\": container with ID starting with bbde0dba90ae568b5e3904137dd0d4afb54cf0d2748dc32b054f7be97fbc12cb not found: ID does not exist"
Oct 30 19:01:14.966897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.966876    5704 scope.go:122] "RemoveContainer" containerID="302cccd1ae6381c400f3482a708bcf3cf6c736f82d811c158c28e5790a8b2f7b"
Oct 30 19:01:14.967131 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:14.967111    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"302cccd1ae6381c400f3482a708bcf3cf6c736f82d811c158c28e5790a8b2f7b\": container with ID starting with 302cccd1ae6381c400f3482a708bcf3cf6c736f82d811c158c28e5790a8b2f7b not found: ID does not exist" containerID="302cccd1ae6381c400f3482a708bcf3cf6c736f82d811c158c28e5790a8b2f7b"
Oct 30 19:01:14.967193 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:14.967134    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"302cccd1ae6381c400f3482a708bcf3cf6c736f82d811c158c28e5790a8b2f7b"} err="failed to get container status \"302cccd1ae6381c400f3482a708bcf3cf6c736f82d811c158c28e5790a8b2f7b\": rpc error: code = NotFound desc = could not find container \"302cccd1ae6381c400f3482a708bcf3cf6c736f82d811c158c28e5790a8b2f7b\": container with ID starting with 302cccd1ae6381c400f3482a708bcf3cf6c736f82d811c158c28e5790a8b2f7b not found: ID does not exist"
Oct 30 19:01:15.154153 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:15.154117    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"]
Oct 30 19:01:15.159595 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:15.159558    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:15.159941 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:15.159925    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ddb8b1dd-openshift-default-857b5876df-4ff45"]
Oct 30 19:01:15.178540 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:15.178509    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e868003e-829c-430f-8888-fb0f7d90c378" path="/var/lib/kubelet/pods/e868003e-829c-430f-8888-fb0f7d90c378/volumes"
Oct 30 19:01:16.954511 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:16.954472    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:17.627787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.627761    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 19:01:17.639743 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.639717    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:17.714205 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.714113    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc" podUID="a2820873-d0e0-4e1a-b227-f73f873ed679" containerName="payload-pre-processing" containerID="cri-o://ea0a3dfe344121415bf5122ceee910287ed31748e3e3b90f22d0c6abdcc49dce" gracePeriod=25
Oct 30 19:01:17.760475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.760447    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-data\") pod \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") "
Oct 30 19:01:17.760645 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.760484    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-credential-socket\") pod \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") "
Oct 30 19:01:17.760645 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.760528    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-certs\") pod \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") "
Oct 30 19:01:17.760645 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.760570    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-socket\") pod \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") "
Oct 30 19:01:17.760645 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.760609    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-envoy\") pod \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") "
Oct 30 19:01:17.760853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.760671    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-kube-api-access-xv6j6\" (UniqueName: \"kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-kube-api-access-xv6j6\") pod \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") "
Oct 30 19:01:17.760853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.760728    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istiod-ca-cert\") pod \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") "
Oct 30 19:01:17.760853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.760774    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-podinfo\") pod \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") "
Oct 30 19:01:17.760853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.760776    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-credential-socket" pod "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" (UID: "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:17.760853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.760781    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-data" pod "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" (UID: "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:17.760853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.760792    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-certs" pod "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" (UID: "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:17.760853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.760807    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-token\" (UniqueName: \"kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-token\") pod \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\" (UID: \"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222\") "
Oct 30 19:01:17.761241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.761007    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-socket" pod "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" (UID: "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:17.761241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.761214    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istiod-ca-cert" pod "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" (UID: "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:17.761354 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.761263    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.761354 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.761284    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.761354 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.761297    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.761354 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.761310    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.763472 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.763444    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-kube-api-access-xv6j6" pod "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" (UID: "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222"). InnerVolumeSpecName "kube-api-access-xv6j6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:17.763886 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.763850    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-envoy" pod "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" (UID: "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:17.763977 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.763940    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-podinfo" pod "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" (UID: "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:01:17.766159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.766136    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-token" pod "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" (UID: "4d79a7ee-5d5e-4fc5-bb91-8b3de8723222"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:17.838867 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.838829    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:17.861959 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.861928    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.862078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.861963    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.862078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.861979    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.862078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.861995    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.862078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.862010    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv6j6\" (UniqueName: \"kubernetes.io/projected/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222-kube-api-access-xv6j6\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:17.941559 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.941536    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"
Oct 30 19:01:17.966222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.966159    5704 generic.go:356] "Generic (PLEG): container finished" podID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerID="b1ee3f6f55cc0b496be99db42fe4de53f26cf0482695eec45ef0818575f0b1f8" exitCode=0
Oct 30 19:01:17.966222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.966199    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"
Oct 30 19:01:17.966638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.966234    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" event={"ID":"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222","Type":"ContainerDied","Data":"b1ee3f6f55cc0b496be99db42fe4de53f26cf0482695eec45ef0818575f0b1f8"}
Oct 30 19:01:17.966638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.966286    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp" event={"ID":"4d79a7ee-5d5e-4fc5-bb91-8b3de8723222","Type":"ContainerDied","Data":"48e0c6b667fbb1453cf57b99022b3601807f9bf3e2eb527b8b1b489c69457fde"}
Oct 30 19:01:17.966638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.966309    5704 scope.go:122] "RemoveContainer" containerID="b1ee3f6f55cc0b496be99db42fe4de53f26cf0482695eec45ef0818575f0b1f8"
Oct 30 19:01:17.967717 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.967694    5704 generic.go:356] "Generic (PLEG): container finished" podID="a2820873-d0e0-4e1a-b227-f73f873ed679" containerID="ea0a3dfe344121415bf5122ceee910287ed31748e3e3b90f22d0c6abdcc49dce" exitCode=0
Oct 30 19:01:17.967815 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.967748    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc" event={"ID":"a2820873-d0e0-4e1a-b227-f73f873ed679","Type":"ContainerDied","Data":"ea0a3dfe344121415bf5122ceee910287ed31748e3e3b90f22d0c6abdcc49dce"}
Oct 30 19:01:17.967815 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.967791    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc" event={"ID":"a2820873-d0e0-4e1a-b227-f73f873ed679","Type":"ContainerDied","Data":"eb956fa1cd1f2671c92d52ea4d728f288e12e5282bcc4683c36356be88ac28d5"}
Oct 30 19:01:17.967924 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.967859    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"
Oct 30 19:01:17.974928 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.974914    5704 scope.go:122] "RemoveContainer" containerID="eaba1cf1b308e1e7126cd9312e20dc8ab3f2c17cb94e4b450edf8faace33e4e2"
Oct 30 19:01:17.984147 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.984128    5704 scope.go:122] "RemoveContainer" containerID="b1ee3f6f55cc0b496be99db42fe4de53f26cf0482695eec45ef0818575f0b1f8"
Oct 30 19:01:17.984394 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:17.984375    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1ee3f6f55cc0b496be99db42fe4de53f26cf0482695eec45ef0818575f0b1f8\": container with ID starting with b1ee3f6f55cc0b496be99db42fe4de53f26cf0482695eec45ef0818575f0b1f8 not found: ID does not exist" containerID="b1ee3f6f55cc0b496be99db42fe4de53f26cf0482695eec45ef0818575f0b1f8"
Oct 30 19:01:17.984450 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.984404    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1ee3f6f55cc0b496be99db42fe4de53f26cf0482695eec45ef0818575f0b1f8"} err="failed to get container status \"b1ee3f6f55cc0b496be99db42fe4de53f26cf0482695eec45ef0818575f0b1f8\": rpc error: code = NotFound desc = could not find container \"b1ee3f6f55cc0b496be99db42fe4de53f26cf0482695eec45ef0818575f0b1f8\": container with ID starting with b1ee3f6f55cc0b496be99db42fe4de53f26cf0482695eec45ef0818575f0b1f8 not found: ID does not exist"
Oct 30 19:01:17.984450 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.984420    5704 scope.go:122] "RemoveContainer" containerID="eaba1cf1b308e1e7126cd9312e20dc8ab3f2c17cb94e4b450edf8faace33e4e2"
Oct 30 19:01:17.984650 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:17.984633    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaba1cf1b308e1e7126cd9312e20dc8ab3f2c17cb94e4b450edf8faace33e4e2\": container with ID starting with eaba1cf1b308e1e7126cd9312e20dc8ab3f2c17cb94e4b450edf8faace33e4e2 not found: ID does not exist" containerID="eaba1cf1b308e1e7126cd9312e20dc8ab3f2c17cb94e4b450edf8faace33e4e2"
Oct 30 19:01:17.984707 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.984651    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaba1cf1b308e1e7126cd9312e20dc8ab3f2c17cb94e4b450edf8faace33e4e2"} err="failed to get container status \"eaba1cf1b308e1e7126cd9312e20dc8ab3f2c17cb94e4b450edf8faace33e4e2\": rpc error: code = NotFound desc = could not find container \"eaba1cf1b308e1e7126cd9312e20dc8ab3f2c17cb94e4b450edf8faace33e4e2\": container with ID starting with eaba1cf1b308e1e7126cd9312e20dc8ab3f2c17cb94e4b450edf8faace33e4e2 not found: ID does not exist"
Oct 30 19:01:17.984707 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.984665    5704 scope.go:122] "RemoveContainer" containerID="ea0a3dfe344121415bf5122ceee910287ed31748e3e3b90f22d0c6abdcc49dce"
Oct 30 19:01:17.991960 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.991945    5704 scope.go:122] "RemoveContainer" containerID="ea0a3dfe344121415bf5122ceee910287ed31748e3e3b90f22d0c6abdcc49dce"
Oct 30 19:01:17.992313 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:17.992293    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea0a3dfe344121415bf5122ceee910287ed31748e3e3b90f22d0c6abdcc49dce\": container with ID starting with ea0a3dfe344121415bf5122ceee910287ed31748e3e3b90f22d0c6abdcc49dce not found: ID does not exist" containerID="ea0a3dfe344121415bf5122ceee910287ed31748e3e3b90f22d0c6abdcc49dce"
Oct 30 19:01:17.992379 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:17.992318    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea0a3dfe344121415bf5122ceee910287ed31748e3e3b90f22d0c6abdcc49dce"} err="failed to get container status \"ea0a3dfe344121415bf5122ceee910287ed31748e3e3b90f22d0c6abdcc49dce\": rpc error: code = NotFound desc = could not find container \"ea0a3dfe344121415bf5122ceee910287ed31748e3e3b90f22d0c6abdcc49dce\": container with ID starting with ea0a3dfe344121415bf5122ceee910287ed31748e3e3b90f22d0c6abdcc49dce not found: ID does not exist"
Oct 30 19:01:18.064622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:18.064598    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a2820873-d0e0-4e1a-b227-f73f873ed679-kube-api-access-g5spl\" (UniqueName: \"kubernetes.io/projected/a2820873-d0e0-4e1a-b227-f73f873ed679-kube-api-access-g5spl\") pod \"a2820873-d0e0-4e1a-b227-f73f873ed679\" (UID: \"a2820873-d0e0-4e1a-b227-f73f873ed679\") "
Oct 30 19:01:18.064758 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:18.064674    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a2820873-d0e0-4e1a-b227-f73f873ed679-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2820873-d0e0-4e1a-b227-f73f873ed679-plugins-config-volume\") pod \"a2820873-d0e0-4e1a-b227-f73f873ed679\" (UID: \"a2820873-d0e0-4e1a-b227-f73f873ed679\") "
Oct 30 19:01:18.065024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:18.065001    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2820873-d0e0-4e1a-b227-f73f873ed679-plugins-config-volume" pod "a2820873-d0e0-4e1a-b227-f73f873ed679" (UID: "a2820873-d0e0-4e1a-b227-f73f873ed679"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:18.066538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:18.066517    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2820873-d0e0-4e1a-b227-f73f873ed679-kube-api-access-g5spl" pod "a2820873-d0e0-4e1a-b227-f73f873ed679" (UID: "a2820873-d0e0-4e1a-b227-f73f873ed679"). InnerVolumeSpecName "kube-api-access-g5spl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:18.166256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:18.166230    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a2820873-d0e0-4e1a-b227-f73f873ed679-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:18.166256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:18.166254    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5spl\" (UniqueName: \"kubernetes.io/projected/a2820873-d0e0-4e1a-b227-f73f873ed679-kube-api-access-g5spl\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:18.183757 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:18.183728    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"]
Oct 30 19:01:18.203858 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:18.203835    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b4e8ca3e-openshift-default-77d86768f5-nk8lp"]
Oct 30 19:01:18.298207 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:18.298178    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:18.380862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:18.380829    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"]
Oct 30 19:01:18.414108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:18.414022    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-df563ff7-9d6f96555-zjbfc"]
Oct 30 19:01:18.584219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:18.584145    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:19.062148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062122    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"]
Oct 30 19:01:19.062551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062538    5704 cpu_manager.go:414] "containerMap: removing container" podUID="a2820873-d0e0-4e1a-b227-f73f873ed679" containerName="payload-pre-processing"
Oct 30 19:01:19.062589 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062551    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a2820873-d0e0-4e1a-b227-f73f873ed679" containerName="payload-pre-processing"
Oct 30 19:01:19.062589 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062577    5704 cpu_manager.go:414] "containerMap: removing container" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy"
Oct 30 19:01:19.062589 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062583    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy"
Oct 30 19:01:19.062688 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062594    5704 cpu_manager.go:414] "containerMap: removing container" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy"
Oct 30 19:01:19.062688 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062599    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy"
Oct 30 19:01:19.062688 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062607    5704 cpu_manager.go:414] "containerMap: removing container" podUID="e868003e-829c-430f-8888-fb0f7d90c378" containerName="istio-proxy"
Oct 30 19:01:19.062688 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062612    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e868003e-829c-430f-8888-fb0f7d90c378" containerName="istio-proxy"
Oct 30 19:01:19.062688 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062676    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="a2820873-d0e0-4e1a-b227-f73f873ed679" containerName="payload-pre-processing"
Oct 30 19:01:19.062688 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062687    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy"
Oct 30 19:01:19.062853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062694    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" containerName="istio-proxy"
Oct 30 19:01:19.062853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062701    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="e868003e-829c-430f-8888-fb0f7d90c378" containerName="istio-proxy"
Oct 30 19:01:19.062853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.062707    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="e868003e-829c-430f-8888-fb0f7d90c378" containerName="istio-proxy"
Oct 30 19:01:19.066137 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.066117    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"
Oct 30 19:01:19.074948 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.074925    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95bhl\" (UniqueName: \"kubernetes.io/projected/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-kube-api-access-95bhl\") pod \"payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx\" (UID: \"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"
Oct 30 19:01:19.075089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.075074    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx\" (UID: \"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"
Oct 30 19:01:19.098124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.098093    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:19.114915 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.114896    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w1-9275c3\""
Oct 30 19:01:19.114915 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.114908    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w1-9275c3-dockercfg-4tl6v\""
Oct 30 19:01:19.125235 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.125212    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"]
Oct 30 19:01:19.175564 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.175538    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95bhl\" (UniqueName: \"kubernetes.io/projected/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-kube-api-access-95bhl\") pod \"payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx\" (UID: \"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"
Oct 30 19:01:19.175684 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.175645    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx\" (UID: \"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"
Oct 30 19:01:19.176322 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.176304    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx\" (UID: \"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"
Oct 30 19:01:19.179076 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.179028    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4d79a7ee-5d5e-4fc5-bb91-8b3de8723222" path="/var/lib/kubelet/pods/4d79a7ee-5d5e-4fc5-bb91-8b3de8723222/volumes"
Oct 30 19:01:19.179509 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.179495    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a2820873-d0e0-4e1a-b227-f73f873ed679" path="/var/lib/kubelet/pods/a2820873-d0e0-4e1a-b227-f73f873ed679/volumes"
Oct 30 19:01:19.232931 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.232906    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95bhl\" (UniqueName: \"kubernetes.io/projected/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-kube-api-access-95bhl\") pod \"payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx\" (UID: \"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"
Oct 30 19:01:19.366382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.366294    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:19.378289 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.378268    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"
Oct 30 19:01:19.750868 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.750835    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"]
Oct 30 19:01:19.758725 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:01:19.758698    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b9a105_e07d_4d0f_8c64_4e4d6c7bc69b.slice/crio-00db81ab8dd93d8dc136658e0e0428e5c4936dd9744bdf764038d877ef778490 WatchSource:0}: Error finding container 00db81ab8dd93d8dc136658e0e0428e5c4936dd9744bdf764038d877ef778490: Status 404 returned error can't find the container with id 00db81ab8dd93d8dc136658e0e0428e5c4936dd9744bdf764038d877ef778490
Oct 30 19:01:19.946526 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.946488    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:19.980698 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.980666    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx" event={"ID":"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b","Type":"ContainerStarted","Data":"8dd731f7e3035379faac95addd792db5037833e98f177fa21d9abec395fd9800"}
Oct 30 19:01:19.980824 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.980734    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx" event={"ID":"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b","Type":"ContainerStarted","Data":"00db81ab8dd93d8dc136658e0e0428e5c4936dd9744bdf764038d877ef778490"}
Oct 30 19:01:19.980824 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:19.980802    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"
Oct 30 19:01:20.129491 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:20.129397    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx" podStartSLOduration=1.129383362 podStartE2EDuration="1.129383362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:20.12879138 +0000 UTC m=+853.074959295" watchObservedRunningTime="2026-10-30 19:01:20.129383362 +0000 UTC m=+853.075551306"
Oct 30 19:01:20.139675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:20.139649    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"]
Oct 30 19:01:20.139871 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:20.139850    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k" podUID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerName="istio-proxy" containerID="cri-o://a9a705e4ee073d9494e4065a5c54a58de51d69f2db0a8def2f7d9c730ccd1b0d" gracePeriod=30
Oct 30 19:01:20.178149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:20.178122    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:20.956312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:20.956275    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:23.001067 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.001022    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:23.193826 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.193791    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:23.387168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.387092    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"]
Oct 30 19:01:23.387507 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.387496    5704 cpu_manager.go:414] "containerMap: removing container" podUID="e868003e-829c-430f-8888-fb0f7d90c378" containerName="istio-proxy"
Oct 30 19:01:23.387545 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.387509    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e868003e-829c-430f-8888-fb0f7d90c378" containerName="istio-proxy"
Oct 30 19:01:23.390009 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.389986    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"
Oct 30 19:01:23.409062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.409016    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:23.421143 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.421118    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57bc5590-a521-4cdd-9f55-7667b13bcfba-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx\" (UID: \"57bc5590-a521-4cdd-9f55-7667b13bcfba\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"
Oct 30 19:01:23.421239 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.421162    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m7w4\" (UniqueName: \"kubernetes.io/projected/57bc5590-a521-4cdd-9f55-7667b13bcfba-kube-api-access-8m7w4\") pod \"payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx\" (UID: \"57bc5590-a521-4cdd-9f55-7667b13bcfba\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"
Oct 30 19:01:23.439253 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.439237    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-22fb18\""
Oct 30 19:01:23.485340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.485305    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-22fb18-dockercfg-74nxs\""
Oct 30 19:01:23.486864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.486833    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"]
Oct 30 19:01:23.522452 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.522426    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57bc5590-a521-4cdd-9f55-7667b13bcfba-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx\" (UID: \"57bc5590-a521-4cdd-9f55-7667b13bcfba\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"
Oct 30 19:01:23.522598 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.522467    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m7w4\" (UniqueName: \"kubernetes.io/projected/57bc5590-a521-4cdd-9f55-7667b13bcfba-kube-api-access-8m7w4\") pod \"payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx\" (UID: \"57bc5590-a521-4cdd-9f55-7667b13bcfba\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"
Oct 30 19:01:23.523169 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.523147    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57bc5590-a521-4cdd-9f55-7667b13bcfba-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx\" (UID: \"57bc5590-a521-4cdd-9f55-7667b13bcfba\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"
Oct 30 19:01:23.541328 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.541305    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"]
Oct 30 19:01:23.544129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.544113    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"
Oct 30 19:01:23.579966 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.579940    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m7w4\" (UniqueName: \"kubernetes.io/projected/57bc5590-a521-4cdd-9f55-7667b13bcfba-kube-api-access-8m7w4\") pod \"payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx\" (UID: \"57bc5590-a521-4cdd-9f55-7667b13bcfba\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"
Oct 30 19:01:23.593825 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.593797    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:23.605828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.605800    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"]
Oct 30 19:01:23.623195 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.623167    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz6xf\" (UniqueName: \"kubernetes.io/projected/1803662c-638a-4442-b019-1ef97209f2a8-kube-api-access-hz6xf\") pod \"payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7\" (UID: \"1803662c-638a-4442-b019-1ef97209f2a8\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"
Oct 30 19:01:23.623366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.623217    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1803662c-638a-4442-b019-1ef97209f2a8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7\" (UID: \"1803662c-638a-4442-b019-1ef97209f2a8\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"
Oct 30 19:01:23.703805 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.703736    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"
Oct 30 19:01:23.724316 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.724290    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz6xf\" (UniqueName: \"kubernetes.io/projected/1803662c-638a-4442-b019-1ef97209f2a8-kube-api-access-hz6xf\") pod \"payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7\" (UID: \"1803662c-638a-4442-b019-1ef97209f2a8\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"
Oct 30 19:01:23.724486 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.724353    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1803662c-638a-4442-b019-1ef97209f2a8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7\" (UID: \"1803662c-638a-4442-b019-1ef97209f2a8\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"
Oct 30 19:01:23.725138 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.725107    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1803662c-638a-4442-b019-1ef97209f2a8-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7\" (UID: \"1803662c-638a-4442-b019-1ef97209f2a8\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"
Oct 30 19:01:23.782798 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.782773    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz6xf\" (UniqueName: \"kubernetes.io/projected/1803662c-638a-4442-b019-1ef97209f2a8-kube-api-access-hz6xf\") pod \"payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7\" (UID: \"1803662c-638a-4442-b019-1ef97209f2a8\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"
Oct 30 19:01:23.789211 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.789140    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:23.852441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.852416    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"]
Oct 30 19:01:23.855839 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.855818    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:23.855980 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.855963    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"
Oct 30 19:01:23.904763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.904731    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-1323a7-openshift-default-dockercfg-2p7xv\""
Oct 30 19:01:23.921502 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.919197    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"]
Oct 30 19:01:23.926440 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.926411    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:23.926575 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.926459    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:23.926575 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.926485    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:23.926575 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.926516    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:23.926575 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.926555    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:23.926790 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.926613    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:23.926790 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.926638    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1b77c27-687a-477c-ab0a-35b33474ec49-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:23.926790 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.926691    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vbpd\" (UniqueName: \"kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-kube-api-access-6vbpd\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:23.926790 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.926732    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:23.968679 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:23.968619    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:24.027439 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.027407    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.027898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.027464    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.027898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.027489    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.027898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.027504    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.027898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.027608    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.027898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.027657    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.027898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.027737    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.027898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.027765    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1b77c27-687a-477c-ab0a-35b33474ec49-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.027898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.027835    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vbpd\" (UniqueName: \"kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-kube-api-access-6vbpd\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.027898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.027868    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.028363 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.027916    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.028363 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.028013    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.028363 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.028249    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.028615 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.028593    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1b77c27-687a-477c-ab0a-35b33474ec49-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.029731 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.029711    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.029941 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.029925    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.078174 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.078145    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"]
Oct 30 19:01:24.085372 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.085355    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.085600 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.085584    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vbpd\" (UniqueName: \"kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-kube-api-access-6vbpd\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.088186 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:01:24.088160    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57bc5590_a521_4cdd_9f55_7667b13bcfba.slice/crio-b7ad1d596ab03c4afd5f1b161286df79e19cad90fa9391527f19d8ceba301b81 WatchSource:0}: Error finding container b7ad1d596ab03c4afd5f1b161286df79e19cad90fa9391527f19d8ceba301b81: Status 404 returned error can't find the container with id b7ad1d596ab03c4afd5f1b161286df79e19cad90fa9391527f19d8ceba301b81
Oct 30 19:01:24.153068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.152993    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:24.179439 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.179416    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:24.232518 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.232489    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"]
Oct 30 19:01:24.242259 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:01:24.242236    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1803662c_638a_4442_b019_1ef97209f2a8.slice/crio-48065517b584cb81d385fcaf623f4503e0ee0c71fd02abc6398a122aadd8e5d4 WatchSource:0}: Error finding container 48065517b584cb81d385fcaf623f4503e0ee0c71fd02abc6398a122aadd8e5d4: Status 404 returned error can't find the container with id 48065517b584cb81d385fcaf623f4503e0ee0c71fd02abc6398a122aadd8e5d4
Oct 30 19:01:24.556118 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.556087    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"]
Oct 30 19:01:24.564797 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:01:24.564772    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1b77c27_687a_477c_ab0a_35b33474ec49.slice/crio-e3a1be4e90b11286229ee34d506578678417df8f534c627c756ed2c038096681 WatchSource:0}: Error finding container e3a1be4e90b11286229ee34d506578678417df8f534c627c756ed2c038096681: Status 404 returned error can't find the container with id e3a1be4e90b11286229ee34d506578678417df8f534c627c756ed2c038096681
Oct 30 19:01:24.567055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.567002    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:24.567113 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.567084    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:24.567145 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:24.567118    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:25.008919 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.008884    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7" event={"ID":"1803662c-638a-4442-b019-1ef97209f2a8","Type":"ContainerStarted","Data":"e6add73921011ae7a832f5fdd6d4ed385ac721e46742ec325bb1ac09f119e13f"}
Oct 30 19:01:25.009098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.008943    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7" event={"ID":"1803662c-638a-4442-b019-1ef97209f2a8","Type":"ContainerStarted","Data":"48065517b584cb81d385fcaf623f4503e0ee0c71fd02abc6398a122aadd8e5d4"}
Oct 30 19:01:25.009098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.009048    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"
Oct 30 19:01:25.010440 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.010417    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx" event={"ID":"57bc5590-a521-4cdd-9f55-7667b13bcfba","Type":"ContainerStarted","Data":"767c04fb670b3932880fb0df8bd509adc25bdb809b40e6eba68771abd3b3f138"}
Oct 30 19:01:25.010571 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.010453    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx" event={"ID":"57bc5590-a521-4cdd-9f55-7667b13bcfba","Type":"ContainerStarted","Data":"b7ad1d596ab03c4afd5f1b161286df79e19cad90fa9391527f19d8ceba301b81"}
Oct 30 19:01:25.010571 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.010498    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"
Oct 30 19:01:25.011860 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.011840    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf" event={"ID":"a1b77c27-687a-477c-ab0a-35b33474ec49","Type":"ContainerStarted","Data":"4e5a90ab0970efdd5afe369eed930a28d7ffe76bfdd2b42ec1cf83fd40a3b277"}
Oct 30 19:01:25.011941 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.011875    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf" event={"ID":"a1b77c27-687a-477c-ab0a-35b33474ec49","Type":"ContainerStarted","Data":"e3a1be4e90b11286229ee34d506578678417df8f534c627c756ed2c038096681"}
Oct 30 19:01:25.120222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.120174    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7" podStartSLOduration=2.12016272 podStartE2EDuration="2.12016272s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:25.119341049 +0000 UTC m=+858.065508952" watchObservedRunningTime="2026-10-30 19:01:25.12016272 +0000 UTC m=+858.066330622"
Oct 30 19:01:25.180442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.180419    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:25.185016 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.184994    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:25.345361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.345016    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx" podStartSLOduration=2.344997937 podStartE2EDuration="2.344997937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:25.343496615 +0000 UTC m=+858.289664517" watchObservedRunningTime="2026-10-30 19:01:25.344997937 +0000 UTC m=+858.291165840"
Oct 30 19:01:25.346473 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.345588    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf" podStartSLOduration=2.345577603 podStartE2EDuration="2.345577603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:25.235184319 +0000 UTC m=+858.181352224" watchObservedRunningTime="2026-10-30 19:01:25.345577603 +0000 UTC m=+858.291745508"
Oct 30 19:01:25.389626 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.389573    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 19:01:25.445908 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.445880    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istiod-ca-cert\") pod \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") "
Oct 30 19:01:25.446018 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.445946    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-envoy\") pod \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") "
Oct 30 19:01:25.446018 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446004    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-credential-socket\") pod \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") "
Oct 30 19:01:25.446106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446048    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-socket\") pod \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") "
Oct 30 19:01:25.446106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446097    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-data\") pod \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") "
Oct 30 19:01:25.446251 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446167    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-certs\") pod \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") "
Oct 30 19:01:25.446251 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446217    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-token\" (UniqueName: \"kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-token\") pod \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") "
Oct 30 19:01:25.446351 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446272    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-podinfo\") pod \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") "
Oct 30 19:01:25.446351 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446323    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-kube-api-access-4tgr7\" (UniqueName: \"kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-kube-api-access-4tgr7\") pod \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\" (UID: \"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc\") "
Oct 30 19:01:25.446351 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446324    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-credential-socket" pod "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" (UID: "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:25.446351 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446340    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-socket" pod "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" (UID: "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:25.446526 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446305    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istiod-ca-cert" pod "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" (UID: "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:01:25.446526 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446430    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-data" pod "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" (UID: "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:25.446526 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446445    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-certs" pod "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" (UID: "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:25.446725 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446704    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:25.446795 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446732    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:25.446795 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446746    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:25.446795 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446761    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:25.446795 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.446774    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:25.448234 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.448208    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-envoy" pod "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" (UID: "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:01:25.448343 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.448312    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-podinfo" pod "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" (UID: "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:01:25.448414 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.448377    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-kube-api-access-4tgr7" pod "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" (UID: "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc"). InnerVolumeSpecName "kube-api-access-4tgr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:25.448482 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.448465    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-token" pod "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" (UID: "0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:01:25.547364 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.547333    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:25.547364 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.547360    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:25.547364 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.547371    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tgr7\" (UniqueName: \"kubernetes.io/projected/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-kube-api-access-4tgr7\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:25.547646 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:25.547379    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:01:26.018765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.018731    5704 generic.go:356] "Generic (PLEG): container finished" podID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerID="a9a705e4ee073d9494e4065a5c54a58de51d69f2db0a8def2f7d9c730ccd1b0d" exitCode=0
Oct 30 19:01:26.018956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.018779    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"
Oct 30 19:01:26.018956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.018837    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k" event={"ID":"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc","Type":"ContainerDied","Data":"a9a705e4ee073d9494e4065a5c54a58de51d69f2db0a8def2f7d9c730ccd1b0d"}
Oct 30 19:01:26.018956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.018880    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k" event={"ID":"0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc","Type":"ContainerDied","Data":"0bdf08240277ce308f23e49fcefeadbbd4fbe085c57ad4fb7aacce85a34d0118"}
Oct 30 19:01:26.018956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.018901    5704 scope.go:122] "RemoveContainer" containerID="a9a705e4ee073d9494e4065a5c54a58de51d69f2db0a8def2f7d9c730ccd1b0d"
Oct 30 19:01:26.019353 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.019327    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:26.020488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.020470    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:01:26.028903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.028887    5704 scope.go:122] "RemoveContainer" containerID="0c8f87ba09d1fe11290e672cd424ca1029da552a93d1d8488856031505f5af2c"
Oct 30 19:01:26.036684 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.036666    5704 scope.go:122] "RemoveContainer" containerID="a9a705e4ee073d9494e4065a5c54a58de51d69f2db0a8def2f7d9c730ccd1b0d"
Oct 30 19:01:26.036976 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:26.036959    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9a705e4ee073d9494e4065a5c54a58de51d69f2db0a8def2f7d9c730ccd1b0d\": container with ID starting with a9a705e4ee073d9494e4065a5c54a58de51d69f2db0a8def2f7d9c730ccd1b0d not found: ID does not exist" containerID="a9a705e4ee073d9494e4065a5c54a58de51d69f2db0a8def2f7d9c730ccd1b0d"
Oct 30 19:01:26.037048 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.036986    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9a705e4ee073d9494e4065a5c54a58de51d69f2db0a8def2f7d9c730ccd1b0d"} err="failed to get container status \"a9a705e4ee073d9494e4065a5c54a58de51d69f2db0a8def2f7d9c730ccd1b0d\": rpc error: code = NotFound desc = could not find container \"a9a705e4ee073d9494e4065a5c54a58de51d69f2db0a8def2f7d9c730ccd1b0d\": container with ID starting with a9a705e4ee073d9494e4065a5c54a58de51d69f2db0a8def2f7d9c730ccd1b0d not found: ID does not exist"
Oct 30 19:01:26.037048 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.037003    5704 scope.go:122] "RemoveContainer" containerID="0c8f87ba09d1fe11290e672cd424ca1029da552a93d1d8488856031505f5af2c"
Oct 30 19:01:26.037282 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:26.037265    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c8f87ba09d1fe11290e672cd424ca1029da552a93d1d8488856031505f5af2c\": container with ID starting with 0c8f87ba09d1fe11290e672cd424ca1029da552a93d1d8488856031505f5af2c not found: ID does not exist" containerID="0c8f87ba09d1fe11290e672cd424ca1029da552a93d1d8488856031505f5af2c"
Oct 30 19:01:26.037338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.037288    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c8f87ba09d1fe11290e672cd424ca1029da552a93d1d8488856031505f5af2c"} err="failed to get container status \"0c8f87ba09d1fe11290e672cd424ca1029da552a93d1d8488856031505f5af2c\": rpc error: code = NotFound desc = could not find container \"0c8f87ba09d1fe11290e672cd424ca1029da552a93d1d8488856031505f5af2c\": container with ID starting with 0c8f87ba09d1fe11290e672cd424ca1029da552a93d1d8488856031505f5af2c not found: ID does not exist"
Oct 30 19:01:26.296655 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.296572    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"]
Oct 30 19:01:26.300671 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:26.300644    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-df563ff7-openshift-default-588695c54c-4qk8k"]
Oct 30 19:01:27.179603 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:27.179574    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" path="/var/lib/kubelet/pods/0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc/volumes"
Oct 30 19:01:28.146857 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.146827    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:28.341112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.341080    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:28.341841 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.341817    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"]
Oct 30 19:01:28.342328 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.342309    5704 cpu_manager.go:414] "containerMap: removing container" podUID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerName="istio-proxy"
Oct 30 19:01:28.342403 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.342328    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerName="istio-proxy"
Oct 30 19:01:28.342479 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.342465    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerName="istio-proxy"
Oct 30 19:01:28.342538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.342487    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerName="istio-proxy"
Oct 30 19:01:28.346234 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.346214    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:28.375773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.375750    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-metrics-certs\") pod \"maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:28.375873 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.375787    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768c017a-29de-4fdc-8782-6924c6740b21-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:28.375873 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.375809    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-maas-api-tls\") pod \"maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:28.376006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.375985    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ljq29\" (UniqueName: \"kubernetes.io/projected/768c017a-29de-4fdc-8782-6924c6740b21-kube-api-access-ljq29\") pod \"maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:28.397734 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.397684    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w3-22fb18-serving-cert\""
Oct 30 19:01:28.445534 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.445511    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"]
Oct 30 19:01:28.477441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.477417    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-metrics-certs\") pod \"maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:28.477574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.477448    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768c017a-29de-4fdc-8782-6924c6740b21-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:28.477574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.477468    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-maas-api-tls\") pod \"maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:28.477574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.477512    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ljq29\" (UniqueName: \"kubernetes.io/projected/768c017a-29de-4fdc-8782-6924c6740b21-kube-api-access-ljq29\") pod \"maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:28.477740 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:28.477630    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-serving-cert: secret "maas-api-e2e-worker-w3-22fb18-serving-cert" not found
Oct 30 19:01:28.477740 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:28.477698    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-maas-api-tls podName:768c017a-29de-4fdc-8782-6924c6740b21 nodeName:}" failed. No retries permitted until 2026-10-30 19:01:28.977678027 +0000 UTC m=+861.923845910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-maas-api-tls") pod "maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc" (UID: "768c017a-29de-4fdc-8782-6924c6740b21") : secret "maas-api-e2e-worker-w3-22fb18-serving-cert" not found
Oct 30 19:01:28.478173 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.478154    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768c017a-29de-4fdc-8782-6924c6740b21-openshift-service-ca\") pod \"maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:28.479860 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.479843    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-metrics-certs\") pod \"maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:28.539712 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.539682    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:28.540508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.540486    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ljq29\" (UniqueName: \"kubernetes.io/projected/768c017a-29de-4fdc-8782-6924c6740b21-kube-api-access-ljq29\") pod \"maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:28.739864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.739769    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:28.919326 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.919293    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:28.981298 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.981272    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-maas-api-tls\") pod \"maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:28.983582 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:28.983555    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-maas-api-tls\") pod \"maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:29.257989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:29.257960    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:29.629618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:29.629540    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"]
Oct 30 19:01:29.637837 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:01:29.637808    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod768c017a_29de_4fdc_8782_6924c6740b21.slice/crio-844b813667c07b935e91ed3edc38e85ae9569c1cd5f4acb51f3fea7a36e9fc76 WatchSource:0}: Error finding container 844b813667c07b935e91ed3edc38e85ae9569c1cd5f4acb51f3fea7a36e9fc76: Status 404 returned error can't find the container with id 844b813667c07b935e91ed3edc38e85ae9569c1cd5f4acb51f3fea7a36e9fc76
Oct 30 19:01:29.914507 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:29.914423    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:01:30.040474 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:30.040439    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc" event={"ID":"768c017a-29de-4fdc-8782-6924c6740b21","Type":"ContainerStarted","Data":"844b813667c07b935e91ed3edc38e85ae9569c1cd5f4acb51f3fea7a36e9fc76"}
Oct 30 19:01:30.983569 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:30.983485    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:30.993061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:30.991223    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"
Oct 30 19:01:31.048716 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:31.048680    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc" event={"ID":"768c017a-29de-4fdc-8782-6924c6740b21","Type":"ContainerStarted","Data":"5fbc336b6494b53dd89d0d71dd19c5c68a8fdc8f23b226596795cfb7dc90b70f"}
Oct 30 19:01:31.048876 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:31.048823    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:31.214741 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:31.214687    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc" podStartSLOduration=2.350756931 podStartE2EDuration="3.2146718s" totalImagesPullingTime="863.914869ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:31.213289492 +0000 UTC m=+864.159457394" watchObservedRunningTime="2026-10-30 19:01:31.2146718 +0000 UTC m=+864.160839703"
Oct 30 19:01:32.495781 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:32.495745    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:32.689195 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:32.689159    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:33.382746 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:33.382702    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:01:33.892804 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:33.892756    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:34.167646 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:34.167562    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:34.373471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:34.373440    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:34.556864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:34.556829    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:36.022508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:36.022477    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"
Oct 30 19:01:36.022982 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:36.022650    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"
Oct 30 19:01:37.061237 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:37.061208    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:01:38.200313 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:38.200281    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:01:39.028845 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:39.028805    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:39.091007 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:39.090979    5704 generic.go:356] "Generic (PLEG): container finished" podID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerID="ca6aa4e2f18fd15e4e12d15b68ae016b72b9919ffaa7d9580feda18e9491392b" exitCode=0
Oct 30 19:01:39.091136 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:39.091022    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j" event={"ID":"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270","Type":"ContainerDied","Data":"ca6aa4e2f18fd15e4e12d15b68ae016b72b9919ffaa7d9580feda18e9491392b"}
Oct 30 19:01:39.091384 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:39.091370    5704 scope.go:122] "RemoveContainer" containerID="ca6aa4e2f18fd15e4e12d15b68ae016b72b9919ffaa7d9580feda18e9491392b"
Oct 30 19:01:39.092558 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:39.092531    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:39.092615 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:39.092598    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:39.092655 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:39.092631    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:01:39.868241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:39.868201    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:40.056302 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.056264    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:40.098445 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.098403    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j" event={"ID":"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270","Type":"ContainerStarted","Data":"728aea0ad48d59e38832e4e768f7080a673a7b8ccba0e1ced675f2e948d98fd3"}
Oct 30 19:01:40.262184 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.262157    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"]
Oct 30 19:01:40.263488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.263455    5704 cpu_manager.go:414] "containerMap: removing container" podUID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerName="istio-proxy"
Oct 30 19:01:40.263488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.263484    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0d1cb2c8-5d6f-40c0-8f7a-df170f7a2fcc" containerName="istio-proxy"
Oct 30 19:01:40.267375 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.267353    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"
Oct 30 19:01:40.317456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.317422    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w2-330025\""
Oct 30 19:01:40.318392 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.318359    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w2-330025-dockercfg-g42nl\""
Oct 30 19:01:40.363221 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.363188    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"]
Oct 30 19:01:40.442058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.436639    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7z59v\" (UniqueName: \"kubernetes.io/projected/b8e3bcdb-a865-428a-8771-6ed558e34369-kube-api-access-7z59v\") pod \"payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79\" (UID: \"b8e3bcdb-a865-428a-8771-6ed558e34369\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"
Oct 30 19:01:40.442058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.436757    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e3bcdb-a865-428a-8771-6ed558e34369-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79\" (UID: \"b8e3bcdb-a865-428a-8771-6ed558e34369\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"
Oct 30 19:01:40.542680 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.542578    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7z59v\" (UniqueName: \"kubernetes.io/projected/b8e3bcdb-a865-428a-8771-6ed558e34369-kube-api-access-7z59v\") pod \"payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79\" (UID: \"b8e3bcdb-a865-428a-8771-6ed558e34369\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"
Oct 30 19:01:40.543021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.542971    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e3bcdb-a865-428a-8771-6ed558e34369-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79\" (UID: \"b8e3bcdb-a865-428a-8771-6ed558e34369\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"
Oct 30 19:01:40.544152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.544127    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e3bcdb-a865-428a-8771-6ed558e34369-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79\" (UID: \"b8e3bcdb-a865-428a-8771-6ed558e34369\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"
Oct 30 19:01:40.601923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.601897    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7z59v\" (UniqueName: \"kubernetes.io/projected/b8e3bcdb-a865-428a-8771-6ed558e34369-kube-api-access-7z59v\") pod \"payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79\" (UID: \"b8e3bcdb-a865-428a-8771-6ed558e34369\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"
Oct 30 19:01:40.688082 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.688022    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:40.689232 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.689207    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 19:01:40.689375 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.689273    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j" podUID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 19:01:40.881274 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:40.881194    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"
Oct 30 19:01:41.274544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:41.274515    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"]
Oct 30 19:01:41.282603 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:01:41.282574    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8e3bcdb_a865_428a_8771_6ed558e34369.slice/crio-036089f7529a252bd63ce7d35745fdfa6ac4849e9d719a81d99e04b2de83b8ce WatchSource:0}: Error finding container 036089f7529a252bd63ce7d35745fdfa6ac4849e9d719a81d99e04b2de83b8ce: Status 404 returned error can't find the container with id 036089f7529a252bd63ce7d35745fdfa6ac4849e9d719a81d99e04b2de83b8ce
Oct 30 19:01:41.687450 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:41.687370    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 19:01:41.687450 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:41.687420    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j" podUID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 19:01:41.800977 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:41.800942    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:42.027867 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:42.027771    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:42.109289 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:42.109257    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79" event={"ID":"b8e3bcdb-a865-428a-8771-6ed558e34369","Type":"ContainerStarted","Data":"4b902dee181dd11b7ed56adb77e8b0956169eace85c2a69cc298bace43f5756f"}
Oct 30 19:01:42.109446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:42.109303    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79" event={"ID":"b8e3bcdb-a865-428a-8771-6ed558e34369","Type":"ContainerStarted","Data":"036089f7529a252bd63ce7d35745fdfa6ac4849e9d719a81d99e04b2de83b8ce"}
Oct 30 19:01:42.109446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:42.109401    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"
Oct 30 19:01:42.687840 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:42.687810    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 19:01:42.688026 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:42.687859    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j" podUID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 19:01:42.942337 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:42.942255    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:43.691766 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.690771    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 19:01:43.691766 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.690844    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j" podUID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 19:01:43.731072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.727639    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79" podStartSLOduration=3.727618059 podStartE2EDuration="3.727618059s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:42.239017719 +0000 UTC m=+875.185185635" watchObservedRunningTime="2026-10-30 19:01:43.727618059 +0000 UTC m=+876.673785962"
Oct 30 19:01:43.731072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.728986    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"]
Oct 30 19:01:43.745071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.742993    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:43.792273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.792245    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-1323a7-serving-cert\""
Oct 30 19:01:43.794113 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.794089    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"]
Oct 30 19:01:43.878107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.878075    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-metrics-certs\") pod \"maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:43.878276 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.878116    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwvfb\" (UniqueName: \"kubernetes.io/projected/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-kube-api-access-lwvfb\") pod \"maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:43.878276 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.878157    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:43.878365 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.878282    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-maas-api-tls\") pod \"maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:43.954240 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.954160    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:43.983017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.982979    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:43.983314 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.983290    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-maas-api-tls\") pod \"maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:43.983447 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.983405    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-metrics-certs\") pod \"maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:43.983447 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.983438    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwvfb\" (UniqueName: \"kubernetes.io/projected/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-kube-api-access-lwvfb\") pod \"maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:43.983573 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:43.983464    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-serving-cert: secret "maas-api-e2e-worker-w4-1323a7-serving-cert" not found
Oct 30 19:01:43.983573 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:43.983541    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-maas-api-tls podName:85b02ffa-cb78-4aa1-8b00-058192c2cb0d nodeName:}" failed. No retries permitted until 2026-10-30 19:01:44.483517811 +0000 UTC m=+877.429685708 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-maas-api-tls") pod "maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm" (UID: "85b02ffa-cb78-4aa1-8b00-058192c2cb0d") : secret "maas-api-e2e-worker-w4-1323a7-serving-cert" not found
Oct 30 19:01:43.983991 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.983970    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:43.986079 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:43.986057    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-metrics-certs\") pod \"maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:44.049780 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:44.049743    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwvfb\" (UniqueName: \"kubernetes.io/projected/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-kube-api-access-lwvfb\") pod \"maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:44.188442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:44.188406    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:44.489946 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:44.489918    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-maas-api-tls\") pod \"maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:44.492359 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:44.492335    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-maas-api-tls\") pod \"maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:44.658386 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:44.658354    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:44.687708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:44.687682    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Oct 30 19:01:44.687826 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:44.687736    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j" podUID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Oct 30 19:01:45.031310 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:45.031277    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"]
Oct 30 19:01:45.040386 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:01:45.040354    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85b02ffa_cb78_4aa1_8b00_058192c2cb0d.slice/crio-70e3e55d2e2d4687e262d011eecb1352ee77215e6a1aec186962d5b1b9bbbc9e WatchSource:0}: Error finding container 70e3e55d2e2d4687e262d011eecb1352ee77215e6a1aec186962d5b1b9bbbc9e: Status 404 returned error can't find the container with id 70e3e55d2e2d4687e262d011eecb1352ee77215e6a1aec186962d5b1b9bbbc9e
Oct 30 19:01:45.128058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:45.127998    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm" event={"ID":"85b02ffa-cb78-4aa1-8b00-058192c2cb0d","Type":"ContainerStarted","Data":"70e3e55d2e2d4687e262d011eecb1352ee77215e6a1aec186962d5b1b9bbbc9e"}
Oct 30 19:01:45.691187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:45.691160    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:45.692164 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:45.692138    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:01:46.135087 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:46.135025    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm" event={"ID":"85b02ffa-cb78-4aa1-8b00-058192c2cb0d","Type":"ContainerStarted","Data":"94d5f96b48283bad5e8ff9e74266eb55b5a624bfef024ddf4bbbbfc0d83dcafc"}
Oct 30 19:01:46.135502 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:46.135208    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:46.246610 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:46.246568    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm" podStartSLOduration=2.395850653 podStartE2EDuration="3.246553193s" totalImagesPullingTime="850.70254ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:46.245309479 +0000 UTC m=+879.191477381" watchObservedRunningTime="2026-10-30 19:01:46.246553193 +0000 UTC m=+879.192721150"
Oct 30 19:01:49.188760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:49.188696    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:52.144966 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:52.144939    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 19:01:52.796088 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:52.796054    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:01:53.117348 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:53.117272    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"
Oct 30 19:01:53.945105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:53.942640    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"]
Oct 30 19:01:53.948470 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:53.948446    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:53.973678 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:53.973653    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-metrics-certs\") pod \"maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:53.973812 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:53.973707    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbrmm\" (UniqueName: \"kubernetes.io/projected/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-kube-api-access-qbrmm\") pod \"maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:53.973812 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:53.973784    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:53.973981 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:53.973960    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-maas-api-tls\") pod \"maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:53.997123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:53.997099    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-330025-serving-cert\""
Oct 30 19:01:54.048268 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.048198    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"]
Oct 30 19:01:54.074528 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.074493    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-maas-api-tls\") pod \"maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:54.074708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.074547    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-metrics-certs\") pod \"maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:54.074708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.074610    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbrmm\" (UniqueName: \"kubernetes.io/projected/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-kube-api-access-qbrmm\") pod \"maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:54.074708 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:54.074635    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-serving-cert: secret "maas-api-e2e-worker-w2-330025-serving-cert" not found
Oct 30 19:01:54.074708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.074651    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:54.074708 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:01:54.074692    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-maas-api-tls podName:fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2 nodeName:}" failed. No retries permitted until 2026-10-30 19:01:54.574676141 +0000 UTC m=+887.520844022 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-maas-api-tls") pod "maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j" (UID: "fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2") : secret "maas-api-e2e-worker-w2-330025-serving-cert" not found
Oct 30 19:01:54.075371 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.075347    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:54.077677 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.077654    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-metrics-certs\") pod \"maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:54.144892 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.144806    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbrmm\" (UniqueName: \"kubernetes.io/projected/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-kube-api-access-qbrmm\") pod \"maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:54.167270 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.167233    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:54.348478 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.348440    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:54.540060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.539430    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:54.581072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.580977    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-maas-api-tls\") pod \"maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:54.583744 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.583705    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-maas-api-tls\") pod \"maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:54.860322 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.860235    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:54.972815 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:54.972787    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:55.247004 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:55.246973    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"]
Oct 30 19:01:55.250078 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:01:55.250019    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc2381bc_b8b5_4d55_8c3c_6df0b2c77fe2.slice/crio-d44544eabde03f644071e9924e71b914cdb47be0f5061727386b8d2c5fd3eea9 WatchSource:0}: Error finding container d44544eabde03f644071e9924e71b914cdb47be0f5061727386b8d2c5fd3eea9: Status 404 returned error can't find the container with id d44544eabde03f644071e9924e71b914cdb47be0f5061727386b8d2c5fd3eea9
Oct 30 19:01:55.575414 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:55.575381    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:55.768659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:55.768624    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:56.192342 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:56.192306    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j" event={"ID":"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2","Type":"ContainerStarted","Data":"32911bc17a18a026aa23b0ba808b5f79580e2347fa75747a317a259dd887fe8d"}
Oct 30 19:01:56.192805 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:56.192365    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j" event={"ID":"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2","Type":"ContainerStarted","Data":"d44544eabde03f644071e9924e71b914cdb47be0f5061727386b8d2c5fd3eea9"}
Oct 30 19:01:56.192805 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:56.192393    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:01:56.344901 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:56.344772    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j" podStartSLOduration=2.540771365 podStartE2EDuration="3.344750208s" totalImagesPullingTime="803.978843ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:56.343053042 +0000 UTC m=+889.289220940" watchObservedRunningTime="2026-10-30 19:01:56.344750208 +0000 UTC m=+889.290918112"
Oct 30 19:01:56.488881 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:56.488734    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:56.707191 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:56.707078    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:56.903784 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:56.903751    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:57.094344 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:57.094305    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:57.273469 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:57.273434    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:59.128431 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:59.128397    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:01:59.600799 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:01:59.600758    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:01.107104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:01.107071    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:02:01.422083 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:01.421995    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:02.202944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:02.202922    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 19:02:02.254949 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:02.254912    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:02.448827 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:02.448794    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:03.233503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:03.233472    5704 generic.go:356] "Generic (PLEG): container finished" podID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerID="4e5a90ab0970efdd5afe369eed930a28d7ffe76bfdd2b42ec1cf83fd40a3b277" exitCode=0
Oct 30 19:02:03.233887 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:03.233543    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf" event={"ID":"a1b77c27-687a-477c-ab0a-35b33474ec49","Type":"ContainerDied","Data":"4e5a90ab0970efdd5afe369eed930a28d7ffe76bfdd2b42ec1cf83fd40a3b277"}
Oct 30 19:02:03.233933 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:03.233883    5704 scope.go:122] "RemoveContainer" containerID="4e5a90ab0970efdd5afe369eed930a28d7ffe76bfdd2b42ec1cf83fd40a3b277"
Oct 30 19:02:03.235128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:03.235095    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:02:03.235233 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:03.235163    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:02:03.235233 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:03.235199    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:02:03.386584 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:02:03.386550    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:02:04.252082 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:04.252047    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf" event={"ID":"a1b77c27-687a-477c-ab0a-35b33474ec49","Type":"ContainerStarted","Data":"ce8cba74dc4ebdbb15e5940855b59b1ebeaaa4cef4f24fe12ea660879a643d8e"}
Oct 30 19:02:04.335488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:04.335449    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:05.179613 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:05.179585    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:02:05.179800 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:05.179748    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 19:02:05.179848 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:05.179793    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf" podUID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 19:02:06.180446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:06.180413    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 19:02:06.180839 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:06.180466    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf" podUID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 19:02:07.180219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:07.180180    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 19:02:07.180397 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:07.180251    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf" podUID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 19:02:08.179785 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:08.179752    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 19:02:08.180289 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:08.179805    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf" podUID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 19:02:08.818981 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:08.818946    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:02:08.998759 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:08.998730    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:09.179999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:09.179916    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Oct 30 19:02:09.179999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:09.179972    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf" podUID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Oct 30 19:02:10.183693 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:10.183664    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:02:10.184127 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:10.184051    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:02:10.184665 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:10.184648    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 19:02:13.615187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:13.615135    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:18.473085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:18.472993    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:18.652087 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:18.652017    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:19.122584 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:19.122555    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:19.308221 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:19.308181    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:19.488536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:19.488455    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:22.792112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:22.792078    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"]
Oct 30 19:02:22.794950 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:22.794930    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:22.843882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:22.843850    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-2301cb-serving-cert\""
Oct 30 19:02:22.891501 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:22.891473    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"]
Oct 30 19:02:22.900676 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:22.900653    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b15a95b6-341d-407c-b8ce-d88a5cf155de-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:22.900773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:22.900690    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-maas-api-tls\") pod \"maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:22.900773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:22.900719    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6vz4\" (UniqueName: \"kubernetes.io/projected/b15a95b6-341d-407c-b8ce-d88a5cf155de-kube-api-access-k6vz4\") pod \"maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:22.900873 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:22.900819    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-metrics-certs\") pod \"maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:23.002251 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.002222    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-metrics-certs\") pod \"maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:23.002420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.002304    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b15a95b6-341d-407c-b8ce-d88a5cf155de-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:23.002420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.002333    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-maas-api-tls\") pod \"maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:23.002420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.002360    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6vz4\" (UniqueName: \"kubernetes.io/projected/b15a95b6-341d-407c-b8ce-d88a5cf155de-kube-api-access-k6vz4\") pod \"maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:23.002594 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:02:23.002471    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-serving-cert: secret "maas-api-e2e-worker-w0-2301cb-serving-cert" not found
Oct 30 19:02:23.002651 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:02:23.002600    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-maas-api-tls podName:b15a95b6-341d-407c-b8ce-d88a5cf155de nodeName:}" failed. No retries permitted until 2026-10-30 19:02:23.502577699 +0000 UTC m=+916.448745594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-maas-api-tls") pod "maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt" (UID: "b15a95b6-341d-407c-b8ce-d88a5cf155de") : secret "maas-api-e2e-worker-w0-2301cb-serving-cert" not found
Oct 30 19:02:23.003083 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.003063    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b15a95b6-341d-407c-b8ce-d88a5cf155de-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:23.004676 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.004649    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-metrics-certs\") pod \"maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:23.013639 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.013612    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:23.062503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.062424    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6vz4\" (UniqueName: \"kubernetes.io/projected/b15a95b6-341d-407c-b8ce-d88a5cf155de-kube-api-access-k6vz4\") pod \"maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:23.203467 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.203418    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:23.392812 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.392733    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:23.508988 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.508951    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-maas-api-tls\") pod \"maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:23.511276 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.511250    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-maas-api-tls\") pod \"maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:23.707293 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.707223    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:23.980879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:23.980775    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:24.082717 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:24.082683    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"]
Oct 30 19:02:24.091324 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:02:24.091292    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb15a95b6_341d_407c_b8ce_d88a5cf155de.slice/crio-90f7634a281f80a8a6331c6d3a1a3c8ea6b733cab447311ef86d6217ceccde9c WatchSource:0}: Error finding container 90f7634a281f80a8a6331c6d3a1a3c8ea6b733cab447311ef86d6217ceccde9c: Status 404 returned error can't find the container with id 90f7634a281f80a8a6331c6d3a1a3c8ea6b733cab447311ef86d6217ceccde9c
Oct 30 19:02:24.355636 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:24.355601    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt" event={"ID":"b15a95b6-341d-407c-b8ce-d88a5cf155de","Type":"ContainerStarted","Data":"90f7634a281f80a8a6331c6d3a1a3c8ea6b733cab447311ef86d6217ceccde9c"}
Oct 30 19:02:25.362275 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:25.362183    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt" event={"ID":"b15a95b6-341d-407c-b8ce-d88a5cf155de","Type":"ContainerStarted","Data":"ebc9dd33cff979d8af14bde7deee9016374d7cb0bb6e05dd1f8b5d2c68cd57a7"}
Oct 30 19:02:25.362275 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:25.362255    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:25.473735 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:25.473663    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt" podStartSLOduration=2.550031492 podStartE2EDuration="3.473645402s" totalImagesPullingTime="923.61391ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:25.473422974 +0000 UTC m=+918.419590879" watchObservedRunningTime="2026-10-30 19:02:25.473645402 +0000 UTC m=+918.419813304"
Oct 30 19:02:28.096435 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:28.096398    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:28.465475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:28.465360    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:30.078873 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:30.078837    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:30.910705 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:30.910675    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:02:31.373374 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:31.373348    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:02:32.893683 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:32.893631    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:33.381891 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:02:33.381862    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:02:37.586092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:37.586056    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:40.876401 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:40.876365    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:02:42.271061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:42.270998    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:46.902526 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:46.902489    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:49.075780 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:49.075738    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:49.254955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:49.254918    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:51.571393 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:51.571322    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:56.182743 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:56.182700    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:57.485967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:57.485934    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:57.902148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:57.902019    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:58.127826 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:58.127786    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:58.312298 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:58.312265    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:02:59.495042 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:02:59.494992    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:01.062257 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:01.062224    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:03.382667 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:03:03.382632    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:03:03.465813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:03.465778    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:03:05.693453 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:05.693419    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:10.271311 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:10.271274    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:14.970180 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:14.970145    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:16.932177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:16.932140    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:19.568959 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:19.568924    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:24.159646 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:24.159617    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:03:24.187415 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:24.187377    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:28.981462 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:28.981430    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:30.388123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:30.388085    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:33.382355 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:03:33.382316    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:03:33.470532 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.470502    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"]
Oct 30 19:03:33.473481 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.473459    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.522508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.522468    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-2301cb\"/\"kube-root-ca.crt\""
Oct 30 19:03:33.568147 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.568124    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2301cb\"/\"default-dockercfg-6c9n4\""
Oct 30 19:03:33.568262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.568153    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-2301cb\"/\"openshift-service-ca.crt\""
Oct 30 19:03:33.568316 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.568273    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2301cb\"/\"simulator-w0-2301cb-kserve-self-signed-certs\""
Oct 30 19:03:33.570158 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.570139    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"]
Oct 30 19:03:33.580741 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.580715    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-home\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.580829 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.580749    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgbxg\" (UniqueName: \"kubernetes.io/projected/1d7b5569-0635-4061-b3d7-eb9890ad2184-kube-api-access-fgbxg\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.581083 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.580836    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-dshm\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.581083 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.580901    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d7b5569-0635-4061-b3d7-eb9890ad2184-tls-certs\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.581083 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.580970    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-tmp-dir\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.581083 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.581078    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-model-cache\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.598771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.598738    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:33.682205 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.682126    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-tmp-dir\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.682205 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.682197    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-model-cache\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.682448 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.682236    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-home\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.682448 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.682254    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgbxg\" (UniqueName: \"kubernetes.io/projected/1d7b5569-0635-4061-b3d7-eb9890ad2184-kube-api-access-fgbxg\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.682448 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.682298    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-dshm\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.682448 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.682321    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d7b5569-0635-4061-b3d7-eb9890ad2184-tls-certs\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.682662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.682546    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-tmp-dir\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.682662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.682609    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-home\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.682742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.682657    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-model-cache\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.684542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.684513    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-dshm\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.684771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.684755    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d7b5569-0635-4061-b3d7-eb9890ad2184-tls-certs\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.740321 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.740290    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgbxg\" (UniqueName: \"kubernetes.io/projected/1d7b5569-0635-4061-b3d7-eb9890ad2184-kube-api-access-fgbxg\") pod \"simulator-w0-2301cb-kserve-577dff5f4d-7h56c\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") " pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:33.785427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:33.785403    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:34.157128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:34.157099    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"]
Oct 30 19:03:34.164381 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:03:34.164357    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d7b5569_0635_4061_b3d7_eb9890ad2184.slice/crio-5b04c41c7b1ebda5bfb40d1e5c776b99ebb13fc506d20dded1c8c7f505fa7a8c WatchSource:0}: Error finding container 5b04c41c7b1ebda5bfb40d1e5c776b99ebb13fc506d20dded1c8c7f505fa7a8c: Status 404 returned error can't find the container with id 5b04c41c7b1ebda5bfb40d1e5c776b99ebb13fc506d20dded1c8c7f505fa7a8c
Oct 30 19:03:34.720303 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:34.720266    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c" event={"ID":"1d7b5569-0635-4061-b3d7-eb9890ad2184","Type":"ContainerStarted","Data":"c7ffc66d6f14f509cb82bfe98b131b53ecdecc3cd891a9cbc372e8e2c8e3a5ec"}
Oct 30 19:03:34.720666 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:34.720314    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c" event={"ID":"1d7b5569-0635-4061-b3d7-eb9890ad2184","Type":"ContainerStarted","Data":"5b04c41c7b1ebda5bfb40d1e5c776b99ebb13fc506d20dded1c8c7f505fa7a8c"}
Oct 30 19:03:34.786417 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:34.786394    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:34.830657 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:34.830609    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c" podStartSLOduration=1.830595532 podStartE2EDuration="1.830595532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:34.829842413 +0000 UTC m=+987.776010316" watchObservedRunningTime="2026-10-30 19:03:34.830595532 +0000 UTC m=+987.776763434"
Oct 30 19:03:38.294554 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:38.294517    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:38.581415 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:38.581324    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:03:38.662909 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:38.662872    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:42.896516 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:42.896482    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:43.786045 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:43.786012    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:43.798309 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:43.798290    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:44.779680 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:44.779643    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:03:47.475061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:47.475014    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:52.081424 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:52.081126    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:56.692366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:56.692238    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:03:57.941409 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:57.941375    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"]
Oct 30 19:03:57.944191 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:57.944175    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:57.992832 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:57.992802    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-22fb18-openshift-default-dockercfg-pqpjp\""
Oct 30 19:03:58.015362 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.015329    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.015510 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.015366    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.015556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.015526    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.015604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.015568    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.015641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.015625    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.015683 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.015652    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.015719 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.015697    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.015763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.015742    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.015799 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.015775    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ww6kw\" (UniqueName: \"kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-kube-api-access-ww6kw\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.049539 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.049505    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"]
Oct 30 19:03:58.116499 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.116466    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.116666 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.116506    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.116666 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.116526    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.116666 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.116555    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.116666 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.116583    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.116666 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.116622    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ww6kw\" (UniqueName: \"kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-kube-api-access-ww6kw\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.116932 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.116678    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.116932 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.116706    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.117091 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.116927    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.117091 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.116958    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.117091 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.116798    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.117376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.117356    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.117484 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.117463    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.117702 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.117684    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.118878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.118855    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.119077 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.119061    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.175167 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.175134    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.175313 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.175256    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ww6kw\" (UniqueName: \"kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-kube-api-access-ww6kw\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.256970 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.256937    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:58.682558 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.682481    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"]
Oct 30 19:03:58.689902 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:03:58.689877    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25293f2e_cd91_42b5_b7b6_8d3568f1bb32.slice/crio-6b1cb552a6c4bc3f7a1941ae3b973c131103dc99f3325f12d1bd6500e99a2d92 WatchSource:0}: Error finding container 6b1cb552a6c4bc3f7a1941ae3b973c131103dc99f3325f12d1bd6500e99a2d92: Status 404 returned error can't find the container with id 6b1cb552a6c4bc3f7a1941ae3b973c131103dc99f3325f12d1bd6500e99a2d92
Oct 30 19:03:58.692181 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.692147    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:03:58.692252 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.692213    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:03:58.692252 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.692241    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:03:58.841889 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.841850    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx" event={"ID":"25293f2e-cd91-42b5-b7b6-8d3568f1bb32","Type":"ContainerStarted","Data":"f60e1c28a175c38ce53bff5b3dff63e61a308162b9420701db457ce90f2ab092"}
Oct 30 19:03:58.842089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.841915    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx" event={"ID":"25293f2e-cd91-42b5-b7b6-8d3568f1bb32","Type":"ContainerStarted","Data":"6b1cb552a6c4bc3f7a1941ae3b973c131103dc99f3325f12d1bd6500e99a2d92"}
Oct 30 19:03:58.955898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:58.955791    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx" podStartSLOduration=1.955776444 podStartE2EDuration="1.955776444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:58.955299076 +0000 UTC m=+1011.901467006" watchObservedRunningTime="2026-10-30 19:03:58.955776444 +0000 UTC m=+1011.901944347"
Oct 30 19:03:59.257753 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:59.257717    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:03:59.258983 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:59.258954    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Oct 30 19:03:59.259129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:59.259001    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx" podUID="25293f2e-cd91-42b5-b7b6-8d3568f1bb32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Oct 30 19:03:59.818384 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:03:59.818355    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:00.258091 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:00.258060    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Oct 30 19:04:00.258465 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:00.258113    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx" podUID="25293f2e-cd91-42b5-b7b6-8d3568f1bb32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Oct 30 19:04:00.944626 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:00.944589    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:04:01.257477 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:01.257449    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Oct 30 19:04:01.257640 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:01.257509    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx" podUID="25293f2e-cd91-42b5-b7b6-8d3568f1bb32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Oct 30 19:04:01.266384 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:01.266354    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:02.257743 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:02.257704    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Oct 30 19:04:02.257928 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:02.257760    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx" podUID="25293f2e-cd91-42b5-b7b6-8d3568f1bb32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Oct 30 19:04:03.257729 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:03.257694    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Oct 30 19:04:03.258148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:03.257743    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx" podUID="25293f2e-cd91-42b5-b7b6-8d3568f1bb32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Oct 30 19:04:03.382890 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:04:03.382858    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:04:04.257639 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:04.257603    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Oct 30 19:04:04.257843 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:04.257662    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx" podUID="25293f2e-cd91-42b5-b7b6-8d3568f1bb32" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Oct 30 19:04:05.282413 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:05.282376    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:04:05.282938 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:05.282822    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:04:05.284538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:05.284518    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:04:05.876447 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:05.876409    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:10.123302 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:10.123263    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:10.465699 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:10.465605    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:15.082843 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:15.082806    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:17.969813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:17.969779    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:18.741123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:18.741090    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:04:19.691815 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:19.691738    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:24.283541 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:24.283495    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:28.960303 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:28.960254    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:33.382572 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:04:33.382534    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:04:33.581563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:33.581525    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:36.399602 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:36.399568    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:37.417058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:37.412679    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:04:38.295140 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:38.295110    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:42.880563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:42.880514    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:47.059228 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:47.059194    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:47.294740 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:47.294711    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:47.673024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:47.672988    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:52.302532 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:52.300559    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:55.145359 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:55.145319    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:04:57.075426 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:57.075391    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:04:57.616894 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:04:57.616857    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:00.168793 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:00.168764    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:01.619856 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:01.619820    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:03.382351 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:05:03.382312    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:05:06.219628 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:06.219585    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:10.821773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:10.821736    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:13.410742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.410708    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"]
Oct 30 19:05:13.413698 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.413677    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.463189 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.463170    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-2301cb-openshift-default-dockercfg-r29tl\""
Oct 30 19:05:13.506645 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.506616    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.506809 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.506666    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.506809 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.506773    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.506906 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.506813    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.506906 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.506831    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcwrg\" (UniqueName: \"kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-kube-api-access-kcwrg\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.506988 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.506944    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.506988 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.506974    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b169d63-af95-42fe-bcca-457d1047fcbf-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.507110 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.507085    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.507199 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.507182    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.516153 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.516132    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"]
Oct 30 19:05:13.608139 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.608105    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.608290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.608167    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.608290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.608218    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.608290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.608244    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcwrg\" (UniqueName: \"kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-kube-api-access-kcwrg\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.608422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.608289    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.608422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.608314    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b169d63-af95-42fe-bcca-457d1047fcbf-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.608422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.608360    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.608422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.608405    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.608636 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.608471    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.608636 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.608591    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.608636 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.608608    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.608789 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.608704    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.608789 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.608778    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.609052 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.609012    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b169d63-af95-42fe-bcca-457d1047fcbf-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.610715 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.610689    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.610889 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.610871    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.665686 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.665637    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.665776 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.665678    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcwrg\" (UniqueName: \"kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-kube-api-access-kcwrg\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:13.726007 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:13.725984    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:14.155822 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:14.155793    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"]
Oct 30 19:05:14.163075 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:05:14.163049    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b169d63_af95_42fe_bcca_457d1047fcbf.slice/crio-d26595ef180a01720155cdad05bb39ff0fd4e3ec4ba7ad3cb4350777d84e3727 WatchSource:0}: Error finding container d26595ef180a01720155cdad05bb39ff0fd4e3ec4ba7ad3cb4350777d84e3727: Status 404 returned error can't find the container with id d26595ef180a01720155cdad05bb39ff0fd4e3ec4ba7ad3cb4350777d84e3727
Oct 30 19:05:14.165233 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:14.165200    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:05:14.165294 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:14.165265    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:05:14.165334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:14.165293    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:05:14.252405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:14.252376    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r" event={"ID":"1b169d63-af95-42fe-bcca-457d1047fcbf","Type":"ContainerStarted","Data":"19274a3d7cf457672269ce51ccf4681eae3703c9b06eff44c95746cb7d530f23"}
Oct 30 19:05:14.252536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:14.252425    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r" event={"ID":"1b169d63-af95-42fe-bcca-457d1047fcbf","Type":"ContainerStarted","Data":"d26595ef180a01720155cdad05bb39ff0fd4e3ec4ba7ad3cb4350777d84e3727"}
Oct 30 19:05:14.737057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:14.733159    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:15.483365 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:15.478199    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:15.736991 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:15.736579    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 19:05:15.736991 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:15.736648    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r" podUID="1b169d63-af95-42fe-bcca-457d1047fcbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 19:05:16.742064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:16.741392    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 19:05:16.742064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:16.741459    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r" podUID="1b169d63-af95-42fe-bcca-457d1047fcbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 19:05:17.740067 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:17.739464    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 19:05:17.740067 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:17.739531    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r" podUID="1b169d63-af95-42fe-bcca-457d1047fcbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 19:05:18.446141 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:18.446083    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r" podStartSLOduration=5.446067941 podStartE2EDuration="5.446067941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:14.366824861 +0000 UTC m=+1087.312992775" watchObservedRunningTime="2026-10-30 19:05:18.446067941 +0000 UTC m=+1091.392235845"
Oct 30 19:05:18.446946 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:18.446928    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:05:18.734522 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:18.734434    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 19:05:18.734769 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:18.734736    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r" podUID="1b169d63-af95-42fe-bcca-457d1047fcbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 19:05:19.729414 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:19.729378    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Oct 30 19:05:19.730004 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:19.729972    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r" podUID="1b169d63-af95-42fe-bcca-457d1047fcbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Oct 30 19:05:20.184575 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:20.183220    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:20.770660 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:20.770634    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:20.771513 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:20.771487    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:20.772706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:20.772683    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:05:24.767071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:24.767021    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:29.332748 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:29.332706    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:33.383024 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:05:33.382989    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:05:33.869428 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:33.869391    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:35.917146 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:35.917115    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:36.109023 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:36.108986    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:36.289045 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:36.288999    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:36.986055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:36.980128    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:37.155703 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:37.155670    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:38.467824 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:38.467782    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:39.949639 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:39.949607    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:43.130207 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:43.125146    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:47.714094 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:47.714052    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:47.945894 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:47.945861    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:48.279912 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:48.279878    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:05:52.487713 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:52.487579    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:56.888355 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:56.888319    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:05:57.114739 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:05:57.114700    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:00.074638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.074608    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"]
Oct 30 19:06:00.088415 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.088384    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.107978 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.107950    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.108130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.108017    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.108130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.108093    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.108217 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.108142    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.108217 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.108186    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.108290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.108235    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.108290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.108255    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.108366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.108288    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwnwg\" (UniqueName: \"kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-kube-api-access-bwnwg\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.108423 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.108401    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.187192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.187166    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"]
Oct 30 19:06:00.209394 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.209367    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.209550 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.209451    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.209550 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.209478    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.209550 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.209506    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.209550 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.209537    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.209765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.209607    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.209765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.209639    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.209765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.209684    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwnwg\" (UniqueName: \"kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-kube-api-access-bwnwg\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.209765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.209743    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.209963 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.209770    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.210020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.209991    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.210105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.210016    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.210144 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.210103    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.210471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.210446    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.212084 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.212061    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.212169 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.212130    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.267456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.267424    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.267592 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.267539    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwnwg\" (UniqueName: \"kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-kube-api-access-bwnwg\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-6mr4h\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.401585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.401503    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:00.506633 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.506588    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:00.836698 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.836666    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"]
Oct 30 19:06:00.843874 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:06:00.843843    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0b50b44_9222_4120_ab91_f11dfd78f7e5.slice/crio-21fa679b87af6d62f3832eb97d0691bf9515360b35c571085948c898452ae929 WatchSource:0}: Error finding container 21fa679b87af6d62f3832eb97d0691bf9515360b35c571085948c898452ae929: Status 404 returned error can't find the container with id 21fa679b87af6d62f3832eb97d0691bf9515360b35c571085948c898452ae929
Oct 30 19:06:00.846299 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.846269    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:00.846378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.846334    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:00.846378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:00.846364    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:01.512979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:01.512944    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" event={"ID":"b0b50b44-9222-4120-ab91-f11dfd78f7e5","Type":"ContainerStarted","Data":"73758a707786321b024ae500946a263af46baa792ba58aa6e98568e2e5e2c931"}
Oct 30 19:06:01.513484 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:01.513006    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" event={"ID":"b0b50b44-9222-4120-ab91-f11dfd78f7e5","Type":"ContainerStarted","Data":"21fa679b87af6d62f3832eb97d0691bf9515360b35c571085948c898452ae929"}
Oct 30 19:06:01.708227 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:01.708175    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" podStartSLOduration=1.708160927 podStartE2EDuration="1.708160927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:01.674082234 +0000 UTC m=+1134.620250136" watchObservedRunningTime="2026-10-30 19:06:01.708160927 +0000 UTC m=+1134.654328829"
Oct 30 19:06:01.809151 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:01.809061    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:02.402463 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:02.402430    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:02.403792 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:02.403768    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 19:06:02.403912 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:02.403823    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" podUID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 19:06:03.382540 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:06:03.382502    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:06:03.401978 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:03.401953    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 19:06:03.402132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:03.402005    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" podUID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 19:06:04.402202 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:04.402167    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 19:06:04.402594 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:04.402228    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" podUID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 19:06:05.402840 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:05.402801    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 19:06:05.403251 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:05.402871    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" podUID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 19:06:06.389344 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:06.389302    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:06.406704 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:06.406670    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 19:06:06.407350 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:06.407299    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" podUID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 19:06:07.402383 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:07.402355    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Oct 30 19:06:07.402533 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:07.402415    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" podUID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Oct 30 19:06:08.541126 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:08.541092    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:06:09.328402 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:09.328362    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:09.329259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:09.329238    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:09.330309 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:09.330286    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:06:11.083147 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:11.082983    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:15.181188 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.181158    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"]
Oct 30 19:06:15.185323 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.185306    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.227350 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.227320    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"]
Oct 30 19:06:15.230336 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.230316    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.291304 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.291268    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"]
Oct 30 19:06:15.317992 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.317965    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz4bq\" (UniqueName: \"kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-kube-api-access-nz4bq\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.318156 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318017    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.318203 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318159    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.318254 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318210    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.318328 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318309    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.318370 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318350    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.318406 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318376    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.318441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318410    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.318441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318432    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.318601 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318582    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.318710 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318657    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrjsp\" (UniqueName: \"kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-kube-api-access-rrjsp\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.318710 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318703    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.318799 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318723    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.318799 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318740    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.318866 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318829    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.318919 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318896    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.318953 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318933    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.318953 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.318949    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.341215 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.341186    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"]
Oct 30 19:06:15.420422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420389    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrjsp\" (UniqueName: \"kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-kube-api-access-rrjsp\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.420422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420426    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.420659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420445    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.420659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420462    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.420659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420485    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.420659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420513    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.420659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420534    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.420659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420550    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.420659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420579    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz4bq\" (UniqueName: \"kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-kube-api-access-nz4bq\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.420659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420598    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.420659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420625    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.420659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420657    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.421198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420689    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.421198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420708    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.421198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420740    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.421198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420765    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.421198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420785    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.421198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.420814    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.421198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.421002    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.421198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.421175    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.421606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.421275    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.421606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.421293    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-credential-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.421606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.421418    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.421606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.421482    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-certs\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.421906 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.421880    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.421988 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.421947    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-socket\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.422141 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.422123    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.422277 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.422122    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-data\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.423812 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.423787    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.424220 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.424176    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.424736 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.424715    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-envoy\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.424963 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.424941    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.478076 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.477989    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.478248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.478230    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz4bq\" (UniqueName: \"kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-kube-api-access-nz4bq\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.478248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.478238    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-token\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-ccr5t\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.478354 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.478339    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrjsp\" (UniqueName: \"kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-kube-api-access-rrjsp\") pod \"maas-default-gateway-openshift-default-8b64f9ff9-4kdgt\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.498090 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.498070    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:15.542329 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.542304    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:15.863826 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.863780    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:15.922339 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.922304    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"]
Oct 30 19:06:15.929332 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:06:15.929302    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d4c56c2_a3e9_4df0_bc32_14dcab3e91d2.slice/crio-8bf89134e2e60d08e80e860ab18160f80011e40a03a95908939e28de1bb10fd6 WatchSource:0}: Error finding container 8bf89134e2e60d08e80e860ab18160f80011e40a03a95908939e28de1bb10fd6: Status 404 returned error can't find the container with id 8bf89134e2e60d08e80e860ab18160f80011e40a03a95908939e28de1bb10fd6
Oct 30 19:06:15.931639 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.931605    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:15.931737 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.931669    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:15.931737 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.931697    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:15.977943 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.977922    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"]
Oct 30 19:06:15.990363 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.990331    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:15.990448 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.990394    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:15.990448 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:15.990424    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:06:16.660604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:16.660566    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" event={"ID":"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2","Type":"ContainerStarted","Data":"3b36a5774c209783b5ce964d2885f1b0f11264897eacc6220c84cc089f236ee5"}
Oct 30 19:06:16.661252 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:16.661209    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" event={"ID":"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2","Type":"ContainerStarted","Data":"8bf89134e2e60d08e80e860ab18160f80011e40a03a95908939e28de1bb10fd6"}
Oct 30 19:06:16.665414 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:16.665389    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" event={"ID":"d494fbe9-42fe-4523-b7c2-a6b1771c18e3","Type":"ContainerStarted","Data":"72e25eec0ef3a4f3f9f6e684854fe0d7b4ede1038b6866212223ed1d02d18edd"}
Oct 30 19:06:16.665627 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:16.665609    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" event={"ID":"d494fbe9-42fe-4523-b7c2-a6b1771c18e3","Type":"ContainerStarted","Data":"c69badfb763e4d79c54720cf4d7b4810bb1022c5a5625cf990613964263e914d"}
Oct 30 19:06:16.778839 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:16.778787    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" podStartSLOduration=1.778776167 podStartE2EDuration="1.778776167s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:16.778549037 +0000 UTC m=+1149.724716943" watchObservedRunningTime="2026-10-30 19:06:16.778776167 +0000 UTC m=+1149.724944070"
Oct 30 19:06:16.893948 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:16.893885    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" podStartSLOduration=1.8938670050000002 podStartE2EDuration="1.893867005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:16.893183948 +0000 UTC m=+1149.839351854" watchObservedRunningTime="2026-10-30 19:06:16.893867005 +0000 UTC m=+1149.840034908"
Oct 30 19:06:17.498376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:17.498336    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:17.500167 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:17.500143    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 19:06:17.500285 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:17.500191    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 19:06:17.546808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:17.546775    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:17.547169 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:17.547147    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 19:06:17.547360 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:17.547337    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 19:06:18.498623 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:18.498588    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 19:06:18.499162 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:18.498645    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 19:06:18.543451 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:18.543413    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 19:06:18.543706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:18.543679    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 19:06:19.499015 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:19.498982    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 19:06:19.499404 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:19.499041    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 19:06:19.547055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:19.546391    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 19:06:19.547055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:19.546464    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 19:06:20.499075 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:20.499017    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 19:06:20.499456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:20.499105    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 19:06:20.544066 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:20.543431    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 19:06:20.544066 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:20.543492    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 19:06:20.581628 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:20.581598    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:21.499117 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:21.499081    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 19:06:21.499507 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:21.499139    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 19:06:21.551056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:21.550563    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 19:06:21.551056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:21.550644    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 19:06:22.498716 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:22.498672    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 19:06:22.498930 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:22.498745    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 19:06:22.542859 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:22.542824    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 19:06:22.543240 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:22.542885    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 19:06:23.499421 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:23.499377    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Oct 30 19:06:23.499622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:23.499444    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Oct 30 19:06:23.542973 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:23.542940    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Oct 30 19:06:23.543345 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:23.543002    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Oct 30 19:06:24.502839 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:24.502810    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:24.503103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:24.503083    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:24.503660 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:24.503644    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:06:24.546662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:24.546636    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:24.547067 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:24.546889    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:24.547640 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:24.547625    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:06:25.271746 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:25.271714    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:30.003095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:30.000759    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:33.382486 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:06:33.382451    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:06:34.572057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:34.570108    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:39.090168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:39.090137    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:39.277195 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:39.277152    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:44.040057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:44.039071    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:48.737620 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:48.737529    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:50.394201 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:50.394115    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:51.431137 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:51.431102    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:06:53.280721 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:53.280667    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:06:57.863248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:06:57.863212    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:00.845695 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:00.845652    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:02.506225 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:02.506185    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:03.384495 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:07:03.384434    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:07:07.064935 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:07.064884    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:10.646897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:10.646861    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:10.827121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:10.827087    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:11.007135 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:11.007101    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:11.684569 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:11.684526    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:12.519925 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:12.519888    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:07:14.832281 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.832254    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"]
Oct 30 19:07:14.836165 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.836147    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:14.899639 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.899612    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-provider-credentials\") pod \"payload-processing-external-model-b4f5657c4-5xnqt\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") " pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:14.899760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.899733    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-plugins-config-volume\") pod \"payload-processing-external-model-b4f5657c4-5xnqt\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") " pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:14.899822 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.899770    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-routing-overlay-volume\") pod \"payload-processing-external-model-b4f5657c4-5xnqt\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") " pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:14.899919 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.899904    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8f2j\" (UniqueName: \"kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-kube-api-access-q8f2j\") pod \"payload-processing-external-model-b4f5657c4-5xnqt\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") " pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:14.930997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.930871    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"kube-root-ca.crt\""
Oct 30 19:07:14.930997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.930925    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"openshift-service-ca.crt\""
Oct 30 19:07:14.930997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.930900    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-dockercfg-bvv24\""
Oct 30 19:07:14.930997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.930951    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-1debe035a4ee\""
Oct 30 19:07:14.930997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.930963    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"routing-overlay\""
Oct 30 19:07:14.931340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.931073    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-e8adb47e7ec7\""
Oct 30 19:07:14.931340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.931087    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"models-as-a-service\"/\"agc-cred-7378d4fb8020\""
Oct 30 19:07:14.931340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.931128    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"models-as-a-service\"/\"payload-processing-external-model-plugins\""
Oct 30 19:07:14.933797 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:14.933776    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"]
Oct 30 19:07:15.003374 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:15.003349    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-plugins-config-volume\") pod \"payload-processing-external-model-b4f5657c4-5xnqt\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") " pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:15.003530 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:15.003380    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-routing-overlay-volume\") pod \"payload-processing-external-model-b4f5657c4-5xnqt\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") " pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:15.003530 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:15.003438    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8f2j\" (UniqueName: \"kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-kube-api-access-q8f2j\") pod \"payload-processing-external-model-b4f5657c4-5xnqt\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") " pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:15.003530 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:15.003474    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-provider-credentials\") pod \"payload-processing-external-model-b4f5657c4-5xnqt\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") " pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:15.004222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:15.004199    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-plugins-config-volume\") pod \"payload-processing-external-model-b4f5657c4-5xnqt\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") " pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:15.004314 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:15.004223    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-routing-overlay-volume\") pod \"payload-processing-external-model-b4f5657c4-5xnqt\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") " pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:15.006225 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:15.006199    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-provider-credentials\") pod \"payload-processing-external-model-b4f5657c4-5xnqt\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") " pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:15.060879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:15.060861    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8f2j\" (UniqueName: \"kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-kube-api-access-q8f2j\") pod \"payload-processing-external-model-b4f5657c4-5xnqt\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") " pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:15.149823 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:15.149767    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:15.521976 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:15.521945    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"]
Oct 30 19:07:15.528651 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:07:15.528624    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73c5f1c3_9998_44b9_8a66_f373989627ad.slice/crio-d1f5a1f80157450bb641bec9dc98fa4a70830bde55034817a82a9002b2451ec6 WatchSource:0}: Error finding container d1f5a1f80157450bb641bec9dc98fa4a70830bde55034817a82a9002b2451ec6: Status 404 returned error can't find the container with id d1f5a1f80157450bb641bec9dc98fa4a70830bde55034817a82a9002b2451ec6
Oct 30 19:07:16.054130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:16.054093    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt" event={"ID":"73c5f1c3-9998-44b9-8a66-f373989627ad","Type":"ContainerStarted","Data":"77290c7a185aedccb46ffa71f2edab525c6d513b152f724f6cf38231deee95e5"}
Oct 30 19:07:16.054501 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:16.054153    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt" event={"ID":"73c5f1c3-9998-44b9-8a66-f373989627ad","Type":"ContainerStarted","Data":"d1f5a1f80157450bb641bec9dc98fa4a70830bde55034817a82a9002b2451ec6"}
Oct 30 19:07:16.054501 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:16.054234    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:16.164507 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:16.164459    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt" podStartSLOduration=2.164446276 podStartE2EDuration="2.164446276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:16.163702838 +0000 UTC m=+1209.109870740" watchObservedRunningTime="2026-10-30 19:07:16.164446276 +0000 UTC m=+1209.110614179"
Oct 30 19:07:16.258537 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:16.258506    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:20.875138 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:20.875084    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:25.550065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:25.549273    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:07:25.550065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:25.549357    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:07:25.598057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:25.597149    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:07:25.598057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:25.597227    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:07:25.785056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:25.784694    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:25.943093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:25.942976    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:07:25.943373 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:25.943327    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:07:26.313054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:26.287451    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:27.063248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:27.063214    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:07:30.466845 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:30.466810    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:30.859839 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:30.859807    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:07:33.495582 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:07:33.495538    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:07:35.215429 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:35.215394    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:39.894058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:39.892243    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:41.276337 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:41.276308    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:44.537950 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:44.537904    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:49.095468 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:49.095428    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:53.036049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:53.036005    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:07:53.665424 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:53.665386    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:07:58.204950 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:07:58.204906    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:00.395529 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:00.395495    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:01.179427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:01.179391    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:02.866475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:02.866442    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:03.381979 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:08:03.381947    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:08:07.476942 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:07.476893    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:12.063120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:12.057387    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:14.182615 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.182585    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"]
Oct 30 19:08:14.186456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.186437    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.247056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.239844    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-home\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.247056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.239942    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-dshm\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.247056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.239968    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-model-cache\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.247056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.240153    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ec41f19-292e-4376-a629-31951649bfe9-tls-certs\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.247056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.240272    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb7bh\" (UniqueName: \"kubernetes.io/projected/8ec41f19-292e-4376-a629-31951649bfe9-kube-api-access-pb7bh\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.247056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.240313    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-tmp-dir\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.282956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.282224    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-1323a7\"/\"distinct-2-w4-1323a7-kserve-self-signed-certs\""
Oct 30 19:08:14.282956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.282259    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-1323a7\"/\"kube-root-ca.crt\""
Oct 30 19:08:14.282956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.282344    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-1323a7\"/\"openshift-service-ca.crt\""
Oct 30 19:08:14.294809 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.294763    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-1323a7\"/\"default-dockercfg-rrhz5\""
Oct 30 19:08:14.295966 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.295927    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"]
Oct 30 19:08:14.315488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.315462    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:14.341900 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.341877    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb7bh\" (UniqueName: \"kubernetes.io/projected/8ec41f19-292e-4376-a629-31951649bfe9-kube-api-access-pb7bh\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.342054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.341921    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-tmp-dir\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.342137 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.342112    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-home\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.342217 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.342198    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-dshm\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.342276 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.342231    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-model-cache\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.342359 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.342347    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ec41f19-292e-4376-a629-31951649bfe9-tls-certs\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.342451 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.342429    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-home\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.342513 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.342234    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-tmp-dir\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.342548 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.342509    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-model-cache\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.344229 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.344209    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-dshm\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.344562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.344545    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ec41f19-292e-4376-a629-31951649bfe9-tls-certs\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.399651 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.399624    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb7bh\" (UniqueName: \"kubernetes.io/projected/8ec41f19-292e-4376-a629-31951649bfe9-kube-api-access-pb7bh\") pod \"distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") " pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.498185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.498161    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:14.540993 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.540962    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:14.899264 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:14.899189    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"]
Oct 30 19:08:14.906404 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:08:14.906378    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8ec41f19_292e_4376_a629_31951649bfe9.slice/crio-ebcd1f18489d148ee86bb4862d9141b6f30657a7996723bf11d4bf3d1c11d802 WatchSource:0}: Error finding container ebcd1f18489d148ee86bb4862d9141b6f30657a7996723bf11d4bf3d1c11d802: Status 404 returned error can't find the container with id ebcd1f18489d148ee86bb4862d9141b6f30657a7996723bf11d4bf3d1c11d802
Oct 30 19:08:15.504345 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:15.504313    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" event={"ID":"8ec41f19-292e-4376-a629-31951649bfe9","Type":"ContainerStarted","Data":"1d4ed510ea862c0036122e42b0f49ab0ab25d41e9fa292977a58361252825259"}
Oct 30 19:08:15.504724 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:15.504359    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" event={"ID":"8ec41f19-292e-4376-a629-31951649bfe9","Type":"ContainerStarted","Data":"ebcd1f18489d148ee86bb4862d9141b6f30657a7996723bf11d4bf3d1c11d802"}
Oct 30 19:08:15.614756 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:15.614700    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" podStartSLOduration=1.614684521 podStartE2EDuration="1.614684521s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:15.613539271 +0000 UTC m=+1268.559707175" watchObservedRunningTime="2026-10-30 19:08:15.614684521 +0000 UTC m=+1268.560852426"
Oct 30 19:08:15.883365 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:15.883273    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:08:16.498463 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:16.498432    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:16.802103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:16.799321    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:21.409753 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:21.409574    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:24.499102 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:24.499070    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:24.511538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:24.511517    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:24.564107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:24.564086    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:08:26.013727 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:26.013691    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:28.413068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:28.382721    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:28.514057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:28.473522    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"]
Oct 30 19:08:28.720779 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:28.720697    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:31.183828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:31.183694    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:33.382838 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:08:33.382802    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:08:33.662314 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:33.662211    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt" podUID="73c5f1c3-9998-44b9-8a66-f373989627ad" containerName="payload-processing" containerID="cri-o://77290c7a185aedccb46ffa71f2edab525c6d513b152f724f6cf38231deee95e5" gracePeriod=25
Oct 30 19:08:33.869205 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:33.869167    5704 generic.go:356] "Generic (PLEG): container finished" podID="73c5f1c3-9998-44b9-8a66-f373989627ad" containerID="77290c7a185aedccb46ffa71f2edab525c6d513b152f724f6cf38231deee95e5" exitCode=0
Oct 30 19:08:33.869385 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:33.869217    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt" event={"ID":"73c5f1c3-9998-44b9-8a66-f373989627ad","Type":"ContainerDied","Data":"77290c7a185aedccb46ffa71f2edab525c6d513b152f724f6cf38231deee95e5"}
Oct 30 19:08:33.912715 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:33.912660    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:08:34.046781 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.046754    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-provider-credentials\" (UniqueName: \"kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-provider-credentials\") pod \"73c5f1c3-9998-44b9-8a66-f373989627ad\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") "
Oct 30 19:08:34.047013 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.046790    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-plugins-config-volume\") pod \"73c5f1c3-9998-44b9-8a66-f373989627ad\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") "
Oct 30 19:08:34.047013 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.046843    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-kube-api-access-q8f2j\" (UniqueName: \"kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-kube-api-access-q8f2j\") pod \"73c5f1c3-9998-44b9-8a66-f373989627ad\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") "
Oct 30 19:08:34.047013 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.046891    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-routing-overlay-volume\") pod \"73c5f1c3-9998-44b9-8a66-f373989627ad\" (UID: \"73c5f1c3-9998-44b9-8a66-f373989627ad\") "
Oct 30 19:08:34.047405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.047344    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-plugins-config-volume" pod "73c5f1c3-9998-44b9-8a66-f373989627ad" (UID: "73c5f1c3-9998-44b9-8a66-f373989627ad"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:34.047405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.047349    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-routing-overlay-volume" pod "73c5f1c3-9998-44b9-8a66-f373989627ad" (UID: "73c5f1c3-9998-44b9-8a66-f373989627ad"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:08:34.047553 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.047463    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:34.049213 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.049189    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-provider-credentials" pod "73c5f1c3-9998-44b9-8a66-f373989627ad" (UID: "73c5f1c3-9998-44b9-8a66-f373989627ad"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:34.049405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.049384    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-kube-api-access-q8f2j" pod "73c5f1c3-9998-44b9-8a66-f373989627ad" (UID: "73c5f1c3-9998-44b9-8a66-f373989627ad"). InnerVolumeSpecName "kube-api-access-q8f2j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:08:34.148801 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.148774    5704 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-provider-credentials\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:34.148801 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.148799    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8f2j\" (UniqueName: \"kubernetes.io/projected/73c5f1c3-9998-44b9-8a66-f373989627ad-kube-api-access-q8f2j\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:34.149087 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.148815    5704 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/73c5f1c3-9998-44b9-8a66-f373989627ad-routing-overlay-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:08:34.875744 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.875710    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt" event={"ID":"73c5f1c3-9998-44b9-8a66-f373989627ad","Type":"ContainerDied","Data":"d1f5a1f80157450bb641bec9dc98fa4a70830bde55034817a82a9002b2451ec6"}
Oct 30 19:08:34.875744 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.875737    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"
Oct 30 19:08:34.876276 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:34.875787    5704 scope.go:122] "RemoveContainer" containerID="77290c7a185aedccb46ffa71f2edab525c6d513b152f724f6cf38231deee95e5"
Oct 30 19:08:35.030329 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:35.030299    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"]
Oct 30 19:08:35.076098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:35.076066    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["models-as-a-service/payload-processing-external-model-b4f5657c4-5xnqt"]
Oct 30 19:08:35.183313 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:35.183236    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="73c5f1c3-9998-44b9-8a66-f373989627ad" path="/var/lib/kubelet/pods/73c5f1c3-9998-44b9-8a66-f373989627ad/volumes"
Oct 30 19:08:35.766290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:35.766258    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:35.966813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:35.966782    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:38.447313 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:38.447276    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:08:40.479919 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:40.479867    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:40.940234 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:40.940182    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:08:40.940545 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:40.940501    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.36:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:08:45.170292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:45.170252    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:49.894115 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:49.894073    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:53.711514 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:53.711456    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:54.642670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:54.642623    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:59.105565 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:59.105529    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:08:59.353232 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:08:59.353201    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:01.510106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:01.510055    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:02.248185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:02.248156    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:09:02.846805 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:02.846762    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:03.385262 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:09:03.385223    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:09:04.097018 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:04.096976    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:08.809806 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:08.809767    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:11.319005 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:11.318965    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:13.422378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:13.422337    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:18.065695 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:18.065655    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:18.852763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:18.852725    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:22.747273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:22.747023    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:27.408376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:27.408340    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:27.601446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:27.601418    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:29.636432 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:29.636399    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:09:32.108596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:32.108555    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:33.382743 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:09:33.382710    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:09:36.390750 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.390721    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"]
Oct 30 19:09:36.391296 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.391281    5704 cpu_manager.go:414] "containerMap: removing container" podUID="73c5f1c3-9998-44b9-8a66-f373989627ad" containerName="payload-processing"
Oct 30 19:09:36.391343 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.391297    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="73c5f1c3-9998-44b9-8a66-f373989627ad" containerName="payload-processing"
Oct 30 19:09:36.391426 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.391417    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="73c5f1c3-9998-44b9-8a66-f373989627ad" containerName="payload-processing"
Oct 30 19:09:36.394693 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.394675    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.456867 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.456826    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bba0064f-500b-4920-88e7-43dc15b56e3e-tls-certs\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.457098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.456881    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqghz\" (UniqueName: \"kubernetes.io/projected/bba0064f-500b-4920-88e7-43dc15b56e3e-kube-api-access-dqghz\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.457098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.457008    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-home\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.457216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.457195    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-dshm\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.457264 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.457246    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-model-cache\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.457364 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.457344    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-tmp-dir\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.490164 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.490134    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2301cb\"/\"distinct-w0-2301cb-kserve-self-signed-certs\""
Oct 30 19:09:36.492367 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.492347    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"]
Oct 30 19:09:36.516530 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.516497    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:36.558750 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.558710    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-tmp-dir\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.569052 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.564234    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bba0064f-500b-4920-88e7-43dc15b56e3e-tls-certs\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.569339 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.569315    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqghz\" (UniqueName: \"kubernetes.io/projected/bba0064f-500b-4920-88e7-43dc15b56e3e-kube-api-access-dqghz\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.569944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.569922    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-home\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.570592 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.570571    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-dshm\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.571142 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.571120    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-model-cache\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.579616 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.560209    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-tmp-dir\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.579828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.579403    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-model-cache\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.579934 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.570390    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-home\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.587412 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.585196    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-dshm\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.587412 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.587395    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bba0064f-500b-4920-88e7-43dc15b56e3e-tls-certs\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.627127 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.627103    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqghz\" (UniqueName: \"kubernetes.io/projected/bba0064f-500b-4920-88e7-43dc15b56e3e-kube-api-access-dqghz\") pod \"distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:36.696977 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.696897    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:36.706826 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:36.706803    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:37.080593 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:37.080563    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"]
Oct 30 19:09:37.087501 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:09:37.087467    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbba0064f_500b_4920_88e7_43dc15b56e3e.slice/crio-60ef52e11003631a87183af6a0cbc0aea2a3086de4bb8f48d228ceb6aee0641c WatchSource:0}: Error finding container 60ef52e11003631a87183af6a0cbc0aea2a3086de4bb8f48d228ceb6aee0641c: Status 404 returned error can't find the container with id 60ef52e11003631a87183af6a0cbc0aea2a3086de4bb8f48d228ceb6aee0641c
Oct 30 19:09:37.323341 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:37.323301    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" event={"ID":"bba0064f-500b-4920-88e7-43dc15b56e3e","Type":"ContainerStarted","Data":"8d5fc9de854a2dc3b211e0c34870340f79af1e78a4de75fd3e187957fed7d5d6"}
Oct 30 19:09:37.323492 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:37.323352    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" event={"ID":"bba0064f-500b-4920-88e7-43dc15b56e3e","Type":"ContainerStarted","Data":"60ef52e11003631a87183af6a0cbc0aea2a3086de4bb8f48d228ceb6aee0641c"}
Oct 30 19:09:37.433910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:37.433806    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" podStartSLOduration=1.4337914999999999 podStartE2EDuration="1.4337915s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:37.432677401 +0000 UTC m=+1350.378845305" watchObservedRunningTime="2026-10-30 19:09:37.4337915 +0000 UTC m=+1350.379959402"
Oct 30 19:09:37.707897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:37.707828    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:41.323071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:41.321278    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:43.329382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.329348    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"]
Oct 30 19:09:43.333726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.333706    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.393937 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.393919    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-330025-openshift-default-dockercfg-9qb6q\""
Oct 30 19:09:43.437747 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.437722    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"]
Oct 30 19:09:43.484128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.484085    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.484272 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.484150    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thgln\" (UniqueName: \"kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-kube-api-access-thgln\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.484272 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.484252    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.484347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.484286    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/307212d3-c2b4-4bc6-908e-607728b8e195-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.484347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.484307    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.484347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.484325    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.484443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.484367    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.484476 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.484458    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.484510 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.484482    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/307212d3-c2b4-4bc6-908e-607728b8e195-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.586079 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.586012    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.586206 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.586082    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thgln\" (UniqueName: \"kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-kube-api-access-thgln\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.586206 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.586109    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.586206 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.586131    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/307212d3-c2b4-4bc6-908e-607728b8e195-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.586206 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.586149    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.586368 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.586265    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.586368 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.586310    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.586485 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.586408    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.586485 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.586435    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/307212d3-c2b4-4bc6-908e-607728b8e195-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.586595 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.586556    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.586670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.586641    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.586730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.586684    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.586730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.586705    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.587100 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.587084    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/307212d3-c2b4-4bc6-908e-607728b8e195-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.588525 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.588496    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.588644 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.588628    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/307212d3-c2b4-4bc6-908e-607728b8e195-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.643058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.643013    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.643418 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.643395    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thgln\" (UniqueName: \"kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-kube-api-access-thgln\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:43.647071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:43.647053    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:44.076530 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:44.076504    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"]
Oct 30 19:09:44.083228 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:09:44.083205    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod307212d3_c2b4_4bc6_908e_607728b8e195.slice/crio-83d137b8c083740759fd6c1346f990d1655239225b91978b1b6473a9b3aeaf72 WatchSource:0}: Error finding container 83d137b8c083740759fd6c1346f990d1655239225b91978b1b6473a9b3aeaf72: Status 404 returned error can't find the container with id 83d137b8c083740759fd6c1346f990d1655239225b91978b1b6473a9b3aeaf72
Oct 30 19:09:44.085471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:44.085441    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:09:44.085537 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:44.085523    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:09:44.085574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:44.085556    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:09:44.364100 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:44.364009    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562" event={"ID":"307212d3-c2b4-4bc6-908e-607728b8e195","Type":"ContainerStarted","Data":"c66ace058f3d7733b1569364c72118482848894dafdce04193369e979c66fbab"}
Oct 30 19:09:44.364100 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:44.364078    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562" event={"ID":"307212d3-c2b4-4bc6-908e-607728b8e195","Type":"ContainerStarted","Data":"83d137b8c083740759fd6c1346f990d1655239225b91978b1b6473a9b3aeaf72"}
Oct 30 19:09:44.647396 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:44.647310    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:45.649823 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:45.649789    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 19:09:45.650238 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:45.649854    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562" podUID="307212d3-c2b4-4bc6-908e-607728b8e195" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 19:09:45.924109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:45.923912    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:46.648181 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:46.648151    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 19:09:46.648373 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:46.648199    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562" podUID="307212d3-c2b4-4bc6-908e-607728b8e195" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 19:09:46.707293 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:46.707257    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:46.744588 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:46.744558    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:46.856587 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:46.856476    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562" podStartSLOduration=3.856451555 podStartE2EDuration="3.856451555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:44.499951742 +0000 UTC m=+1357.446119645" watchObservedRunningTime="2026-10-30 19:09:46.856451555 +0000 UTC m=+1359.802619459"
Oct 30 19:09:47.391728 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:47.391696    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:09:47.648564 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:47.648470    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 19:09:47.648564 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:47.648546    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562" podUID="307212d3-c2b4-4bc6-908e-607728b8e195" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 19:09:48.647918 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:48.647884    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 19:09:48.648319 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:48.647935    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562" podUID="307212d3-c2b4-4bc6-908e-607728b8e195" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 19:09:48.659665 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:48.659627    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:49.306880 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:49.306846    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:49.479618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:49.479583    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:49.647534 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:49.647454    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 19:09:49.647534 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:49.647508    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562" podUID="307212d3-c2b4-4bc6-908e-607728b8e195" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 19:09:49.665470 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:49.665439    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:50.647869 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:50.647828    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Oct 30 19:09:50.648089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:50.647906    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562" podUID="307212d3-c2b4-4bc6-908e-607728b8e195" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Oct 30 19:09:50.684318 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:50.684287    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:09:51.652065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:51.652021    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:51.652334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:51.652314    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:51.652910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:51.652886    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:09:55.282638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:09:55.282607    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:00.005745 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:00.005706    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:01.690532 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:01.690499    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:10:03.107498 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:03.107458    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:03.383064 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:10:03.382966    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:10:04.203697 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:04.203663    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:04.393175 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:04.393146    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:04.607145 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:04.602090    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:09.199387 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:09.199297    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:13.800912 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:13.800870    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:18.480016 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:18.479966    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:20.635218 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:20.635179    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:20.939132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:20.938841    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:23.092690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:23.090211    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:27.708223 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:27.708182    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:28.178516 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:28.178481    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:10:32.470849 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:32.470816    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:33.383549 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:10:33.383510    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:10:37.154795 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:37.154749    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:41.731508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:41.731466    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:43.811985 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:43.811945    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"]
Oct 30 19:10:43.816369 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:43.816347    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:43.943750 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:43.943713    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:43.943922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:43.943776    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:43.943995 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:43.943940    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:43.943995 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:43.943981    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:43.944118 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:43.944022    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:43.944198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:43.944167    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wr87\" (UniqueName: \"kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-kube-api-access-6wr87\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:43.944243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:43.944214    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:43.944382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:43.944365    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28fe6291-de37-42c5-abe2-76e43b0f89b3-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:43.944446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:43.944431    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.045566 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.045531    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.045566 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.045569    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.045808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.045600    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.045808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.045730    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wr87\" (UniqueName: \"kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-kube-api-access-6wr87\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.045808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.045780    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.045971 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.045887    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28fe6291-de37-42c5-abe2-76e43b0f89b3-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.045971 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.045952    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.046114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.045990    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.046114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.046057    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.046114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.046100    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.046280 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.046110    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.046442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.046418    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.046623 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.046554    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.046753 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.046729    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28fe6291-de37-42c5-abe2-76e43b0f89b3-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.048280 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.048256    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.048885 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.048865    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.064325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.064261    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"]
Oct 30 19:10:44.107124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.107099    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wr87\" (UniqueName: \"kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-kube-api-access-6wr87\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.109170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.109147    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.129857 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.129837    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:44.565868 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.565836    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"]
Oct 30 19:10:44.571917 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:10:44.571893    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28fe6291_de37_42c5_abe2_76e43b0f89b3.slice/crio-155d8c8b897fc675caa7747b154a1d68bd1aa42ff96377f0b29e9e2d23408cb9 WatchSource:0}: Error finding container 155d8c8b897fc675caa7747b154a1d68bd1aa42ff96377f0b29e9e2d23408cb9: Status 404 returned error can't find the container with id 155d8c8b897fc675caa7747b154a1d68bd1aa42ff96377f0b29e9e2d23408cb9
Oct 30 19:10:44.574347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.574316    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:44.574423 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.574378    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:44.574423 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.574405    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:44.706108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.706067    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" event={"ID":"28fe6291-de37-42c5-abe2-76e43b0f89b3","Type":"ContainerStarted","Data":"f8ceadf9b2dc93e16c6422873275cf971130a5c32e02113f66377fb3c991f4b9"}
Oct 30 19:10:44.706294 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.706130    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" event={"ID":"28fe6291-de37-42c5-abe2-76e43b0f89b3","Type":"ContainerStarted","Data":"155d8c8b897fc675caa7747b154a1d68bd1aa42ff96377f0b29e9e2d23408cb9"}
Oct 30 19:10:44.823080 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:44.822953    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" podStartSLOduration=1.822936346 podStartE2EDuration="1.822936346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:44.818505922 +0000 UTC m=+1417.764673831" watchObservedRunningTime="2026-10-30 19:10:44.822936346 +0000 UTC m=+1417.769104251"
Oct 30 19:10:45.130606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:45.130538    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:45.132078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:45.132027    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:10:45.132190 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:45.132098    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" podUID="28fe6291-de37-42c5-abe2-76e43b0f89b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 19:10:46.092001 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.091963    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"]
Oct 30 19:10:46.096212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.096195    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.144510 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.144470    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:10:46.144720 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.144554    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" podUID="28fe6291-de37-42c5-abe2-76e43b0f89b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 19:10:46.174752 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.174723    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.174910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.174765    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.174910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.174840    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.175049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.174939    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.175049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.174981    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.175049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.175016    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.175189 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.175168    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt9wf\" (UniqueName: \"kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-kube-api-access-vt9wf\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.175232 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.175200    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.175276 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.175242    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.192073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.192027    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"]
Oct 30 19:10:46.276388 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.276351    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.276583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.276396    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.276583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.276433    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.276583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.276516    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt9wf\" (UniqueName: \"kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-kube-api-access-vt9wf\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.276583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.276562    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.276810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.276628    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.276810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.276686    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.276810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.276731    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.276810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.276805    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.277236 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.277211    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.277401 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.277300    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.277536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.277314    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.277652 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.277534    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.277763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.277556    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.279701 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.279680    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.279928 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.279907    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.347511 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.347436    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt9wf\" (UniqueName: \"kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-kube-api-access-vt9wf\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.347741 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.347714    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.410188 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.410153    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:46.437055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.436450    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:46.945270 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.945235    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"]
Oct 30 19:10:46.949463 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:10:46.949436    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34886f9b_18a4_4889_bef9_5b1c16f8ce2e.slice/crio-d932746dbe7c01a2e5fb842b10b30fc9dd0662cd9aedbf72bdcb91d82fd27f8c WatchSource:0}: Error finding container d932746dbe7c01a2e5fb842b10b30fc9dd0662cd9aedbf72bdcb91d82fd27f8c: Status 404 returned error can't find the container with id d932746dbe7c01a2e5fb842b10b30fc9dd0662cd9aedbf72bdcb91d82fd27f8c
Oct 30 19:10:46.951985 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.951951    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:46.952114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.952028    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:46.952114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:46.952098    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:10:47.138796 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:47.138380    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:10:47.138796 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:47.138433    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" podUID="28fe6291-de37-42c5-abe2-76e43b0f89b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 19:10:47.747059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:47.746385    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72" event={"ID":"34886f9b-18a4-4889-bef9-5b1c16f8ce2e","Type":"ContainerStarted","Data":"527d6d0c7656f7581f4018e37dc397f4b5a3c27701101c7288fee6f4dc3699b7"}
Oct 30 19:10:47.747059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:47.746459    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72" event={"ID":"34886f9b-18a4-4889-bef9-5b1c16f8ce2e","Type":"ContainerStarted","Data":"d932746dbe7c01a2e5fb842b10b30fc9dd0662cd9aedbf72bdcb91d82fd27f8c"}
Oct 30 19:10:47.965079 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:47.962734    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72" podStartSLOduration=1.962711785 podStartE2EDuration="1.962711785s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:47.90885909 +0000 UTC m=+1420.855026994" watchObservedRunningTime="2026-10-30 19:10:47.962711785 +0000 UTC m=+1420.908879690"
Oct 30 19:10:48.134111 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:48.130371    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:10:48.134111 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:48.130438    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" podUID="28fe6291-de37-42c5-abe2-76e43b0f89b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 19:10:48.410878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:48.410793    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:48.412425 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:48.412398    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 19:10:48.412549 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:48.412447    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72" podUID="34886f9b-18a4-4889-bef9-5b1c16f8ce2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 19:10:49.136057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:49.133422    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:10:49.136057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:49.133486    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" podUID="28fe6291-de37-42c5-abe2-76e43b0f89b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 19:10:49.411586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:49.411505    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 19:10:49.411586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:49.411565    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72" podUID="34886f9b-18a4-4889-bef9-5b1c16f8ce2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 19:10:50.146057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:50.145902    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:10:50.146057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:50.145974    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" podUID="28fe6291-de37-42c5-abe2-76e43b0f89b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 19:10:50.411607 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:50.411501    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 19:10:50.412226 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:50.412179    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72" podUID="34886f9b-18a4-4889-bef9-5b1c16f8ce2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 19:10:51.131605 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:51.131566    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:10:51.131941 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:51.131886    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" podUID="28fe6291-de37-42c5-abe2-76e43b0f89b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 19:10:51.410931 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:51.410846    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 19:10:51.411161 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:51.411140    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72" podUID="34886f9b-18a4-4889-bef9-5b1c16f8ce2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 19:10:52.138423 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:52.138385    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Oct 30 19:10:52.139017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:52.138983    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" podUID="28fe6291-de37-42c5-abe2-76e43b0f89b3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Oct 30 19:10:52.410875 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:52.410801    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 19:10:52.410875 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:52.410859    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72" podUID="34886f9b-18a4-4889-bef9-5b1c16f8ce2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 19:10:52.541808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:52.541710    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:53.144778 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:53.144747    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:53.145290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:53.145257    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:53.146133 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:53.146107    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:10:53.410560 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:53.410482    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Oct 30 19:10:53.410742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:53.410535    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72" podUID="34886f9b-18a4-4889-bef9-5b1c16f8ce2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Oct 30 19:10:54.681621 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:54.681592    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:54.682214 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:54.682194    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:54.682986 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:54.682966    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:10:57.459490 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:57.459455    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:10:59.990313 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:59.990277    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"]
Oct 30 19:10:59.994234 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:10:59.994214    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.098729 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.098698    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"]
Oct 30 19:11:00.109966 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.109934    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4827e66-1986-4644-bfa8-9741b5c53cd3-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.110151 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.109984    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.110151 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.110064    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.110151 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.110131    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.110328 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.110216    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.110328 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.110258    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fsjr6\" (UniqueName: \"kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-kube-api-access-fsjr6\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.110328 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.110287    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.110485 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.110343    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.110485 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.110387    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.211790 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.211759    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4827e66-1986-4644-bfa8-9741b5c53cd3-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.211790 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.211793    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.211997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.211812    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.211997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.211847    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.211997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.211874    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.211997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.211892    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fsjr6\" (UniqueName: \"kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-kube-api-access-fsjr6\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.211997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.211910    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.211997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.211943    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.211997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.211979    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.212391 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.212253    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.212391 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.212326    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.212391 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.212368    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.212549 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.212512    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.212640 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.212619    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4827e66-1986-4644-bfa8-9741b5c53cd3-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.214269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.214245    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.214409 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.214395    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.269915 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.269887    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fsjr6\" (UniqueName: \"kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-kube-api-access-fsjr6\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.269915 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.269907    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.308518 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.308497    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:00.740578 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.740506    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"]
Oct 30 19:11:00.747103 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:11:00.747076    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4827e66_1986_4644_bfa8_9741b5c53cd3.slice/crio-193b883e7beb3b5ce9c8d8e8ee3b1d4de987104f1804f403334a9335cd621b57 WatchSource:0}: Error finding container 193b883e7beb3b5ce9c8d8e8ee3b1d4de987104f1804f403334a9335cd621b57: Status 404 returned error can't find the container with id 193b883e7beb3b5ce9c8d8e8ee3b1d4de987104f1804f403334a9335cd621b57
Oct 30 19:11:00.749477 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.749437    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:00.749559 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.749502    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:00.749559 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.749531    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:00.915739 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.915697    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" event={"ID":"f4827e66-1986-4644-bfa8-9741b5c53cd3","Type":"ContainerStarted","Data":"222d7f606a99067fefda53971f15afd26ff33438ce59bc6594e910e4af4a60f7"}
Oct 30 19:11:00.915933 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:00.915767    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" event={"ID":"f4827e66-1986-4644-bfa8-9741b5c53cd3","Type":"ContainerStarted","Data":"193b883e7beb3b5ce9c8d8e8ee3b1d4de987104f1804f403334a9335cd621b57"}
Oct 30 19:11:01.033196 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:01.033150    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" podStartSLOduration=2.033136017 podStartE2EDuration="2.033136017s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:01.030772146 +0000 UTC m=+1433.976940073" watchObservedRunningTime="2026-10-30 19:11:01.033136017 +0000 UTC m=+1433.979303920"
Oct 30 19:11:01.309420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:01.309346    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:01.310778 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:01.310752    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 19:11:01.310900 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:01.310798    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 19:11:02.055529 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:02.055492    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:11:02.063225 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:02.063193    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:02.309766 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:02.309691    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 19:11:02.309766 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:02.309743    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 19:11:03.045055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.035599    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:03.316116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.312759    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 19:11:03.316116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.312820    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 19:11:03.410143 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:11:03.410100    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:11:03.500056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.499227    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"]
Oct 30 19:11:03.504182 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.504160    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.539716 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.539678    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-home\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.540089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.540063    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-tmp-dir\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.540338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.540319    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e84096c6-01ea-407c-8d70-46f7b14bde3f-tls-certs\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.541700 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.541680    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-dshm\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.541891 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.541870    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx9cl\" (UniqueName: \"kubernetes.io/projected/e84096c6-01ea-407c-8d70-46f7b14bde3f-kube-api-access-lx9cl\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.551049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.542013    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-model-cache\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.612635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.610708    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-2301cb\"/\"distinct-2-w0-2301cb-kserve-self-signed-certs\""
Oct 30 19:11:03.615014 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.614990    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"]
Oct 30 19:11:03.616715 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.616691    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:03.655802 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.655756    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-dshm\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.656600 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.656578    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx9cl\" (UniqueName: \"kubernetes.io/projected/e84096c6-01ea-407c-8d70-46f7b14bde3f-kube-api-access-lx9cl\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.657535 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.657512    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-model-cache\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.657724 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.657700    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-model-cache\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.657972 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.657955    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-home\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.658538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.658521    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-tmp-dir\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.659018 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.658999    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e84096c6-01ea-407c-8d70-46f7b14bde3f-tls-certs\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.669800 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.658872    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-tmp-dir\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.669978 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.658351    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-home\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.689841 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.689811    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-dshm\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.735254 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.735192    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e84096c6-01ea-407c-8d70-46f7b14bde3f-tls-certs\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.809063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.807754    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx9cl\" (UniqueName: \"kubernetes.io/projected/e84096c6-01ea-407c-8d70-46f7b14bde3f-kube-api-access-lx9cl\") pod \"distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") " pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:03.855310 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:03.855060    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:04.375062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:04.372183    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 19:11:04.375062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:04.372253    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 19:11:04.642110 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:04.640473    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"]
Oct 30 19:11:04.995070 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:04.994659    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl" event={"ID":"e84096c6-01ea-407c-8d70-46f7b14bde3f","Type":"ContainerStarted","Data":"e4473e938b1d94f74460f48fea3d6f3effd526d4467694c2ea1ff693574fa985"}
Oct 30 19:11:04.995070 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:04.994712    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl" event={"ID":"e84096c6-01ea-407c-8d70-46f7b14bde3f","Type":"ContainerStarted","Data":"dcec367c545932d636d385c7d60f49bef05c0b6087d23d662245a68275c1fa03"}
Oct 30 19:11:05.109788 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:05.109718    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl" podStartSLOduration=2.109700238 podStartE2EDuration="2.109700238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:05.107391265 +0000 UTC m=+1438.053559168" watchObservedRunningTime="2026-10-30 19:11:05.109700238 +0000 UTC m=+1438.055868143"
Oct 30 19:11:05.309302 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:05.309267    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 19:11:05.309471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:05.309315    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 19:11:05.859495 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:05.859461    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:06.309202 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:06.309173    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 19:11:06.309378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:06.309222    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 19:11:06.888124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:06.888077    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:07.309687 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:07.309653    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Oct 30 19:11:07.309884 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:07.309699    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Oct 30 19:11:09.351187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:09.351143    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:11:09.351620 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:09.351204    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:09.494014 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:09.493984    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:09.494385 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:09.494364    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:09.495028 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:09.495012    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:11:11.584045 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:11.583997    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:13.859006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:13.858975    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:13.871475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:13.871455    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:14.057601 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:14.057573    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:16.206713 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.206673    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:16.346877 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.346843    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"]
Oct 30 19:11:16.353422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.353392    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.440347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.440299    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkxjd\" (UniqueName: \"kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-kube-api-access-jkxjd\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.440531 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.440369    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42b62545-91fd-4215-9c57-b1273c5471d6-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.440531 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.440399    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.440531 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.440429    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.440531 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.440498    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.440771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.440580    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.440771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.440634    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.440771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.440712    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42b62545-91fd-4215-9c57-b1273c5471d6-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.440771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.440758    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.450843 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.450815    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"]
Oct 30 19:11:16.575857 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.575808    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.576217 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.576187    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42b62545-91fd-4215-9c57-b1273c5471d6-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.576396 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.576375    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.576571 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.576557    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkxjd\" (UniqueName: \"kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-kube-api-access-jkxjd\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.576706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.576692    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42b62545-91fd-4215-9c57-b1273c5471d6-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.576809 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.576795    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.576920 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.576906    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.581146 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.581115    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.581413 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.581389    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.582082 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.582058    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.769060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.768524    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-envoy\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.774519 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.774486    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42b62545-91fd-4215-9c57-b1273c5471d6-istiod-ca-cert\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.776230 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.775854    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-data\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.776683 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.776663    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-certs\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.777400 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.777367    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-credential-socket\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.793477 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.778229    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42b62545-91fd-4215-9c57-b1273c5471d6-istio-podinfo\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.904466 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.904355    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-istio-token\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.965092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.956284    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkxjd\" (UniqueName: \"kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-kube-api-access-jkxjd\") pod \"e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") " pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:16.973559 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:16.973281    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:17.691585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:17.691332    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"]
Oct 30 19:11:17.696800 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:11:17.696718    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42b62545_91fd_4215_9c57_b1273c5471d6.slice/crio-8b88ae5024f0984985326058ace44ce2301e8a0705a00444dba99434c4a77023 WatchSource:0}: Error finding container 8b88ae5024f0984985326058ace44ce2301e8a0705a00444dba99434c4a77023: Status 404 returned error can't find the container with id 8b88ae5024f0984985326058ace44ce2301e8a0705a00444dba99434c4a77023
Oct 30 19:11:17.701057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:17.700993    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:17.701170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:17.701125    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:17.701232 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:17.701173    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:18.123120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:18.122002    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl" event={"ID":"42b62545-91fd-4215-9c57-b1273c5471d6","Type":"ContainerStarted","Data":"e98cf09b68dbedca9917bafb8508794bd0b311300c5b0485b139763920de2300"}
Oct 30 19:11:18.123120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:18.122103    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl" event={"ID":"42b62545-91fd-4215-9c57-b1273c5471d6","Type":"ContainerStarted","Data":"8b88ae5024f0984985326058ace44ce2301e8a0705a00444dba99434c4a77023"}
Oct 30 19:11:18.974177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:18.974145    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:18.975692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:18.975663    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 19:11:18.975804 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:18.975726    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl" podUID="42b62545-91fd-4215-9c57-b1273c5471d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 19:11:19.974790 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:19.974761    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 19:11:19.975170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:19.974810    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl" podUID="42b62545-91fd-4215-9c57-b1273c5471d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 19:11:20.974384 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:20.974350    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 19:11:20.974652 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:20.974394    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl" podUID="42b62545-91fd-4215-9c57-b1273c5471d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 19:11:21.418840 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:21.418806    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:21.974824 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:21.974791    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 19:11:21.974996 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:21.974841    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl" podUID="42b62545-91fd-4215-9c57-b1273c5471d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 19:11:22.974850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:22.974810    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 19:11:22.975282 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:22.974869    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl" podUID="42b62545-91fd-4215-9c57-b1273c5471d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 19:11:23.974709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:23.974673    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Oct 30 19:11:23.974916 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:23.974731    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl" podUID="42b62545-91fd-4215-9c57-b1273c5471d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Oct 30 19:11:25.387166 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:25.387129    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:25.387649 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:25.387535    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:25.388818 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:25.388798    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:11:25.550375 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:25.550325    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl" podStartSLOduration=9.550312241 podStartE2EDuration="9.550312241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:18.248912775 +0000 UTC m=+1451.195080675" watchObservedRunningTime="2026-10-30 19:11:25.550312241 +0000 UTC m=+1458.496480219"
Oct 30 19:11:26.268774 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:26.268737    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:26.729427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:26.729331    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:26.930536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:26.930508    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:28.104004 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:28.103949    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:28.300861 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:28.300830    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:31.120889 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.120846    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:31.342923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.342893    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"]
Oct 30 19:11:31.348067 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.348010    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.452274 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.452183    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.452437 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.452282    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d9ad6f-a16b-4647-a761-a417921cca9a-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.452726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.452690    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.452808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.452739    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.452808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.452770    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.452938 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.452857    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.452938 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.452894    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.453105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.453086    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgh2v\" (UniqueName: \"kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-kube-api-access-bgh2v\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.453180 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.453133    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.458348 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.458315    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"]
Oct 30 19:11:31.554355 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.554322    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.554562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.554374    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.554562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.554411    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.554562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.554481    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.554562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.554510    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.554803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.554621    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgh2v\" (UniqueName: \"kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-kube-api-access-bgh2v\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.554803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.554672    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.554803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.554751    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.554803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.554761    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.554803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.554774    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d9ad6f-a16b-4647-a761-a417921cca9a-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.555165 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.554804    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.555478 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.555436    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d9ad6f-a16b-4647-a761-a417921cca9a-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.555638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.555532    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.555826 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.555803    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.557762 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.557744    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.558124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.558101    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.612488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.612458    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.613373 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.613356    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgh2v\" (UniqueName: \"kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-kube-api-access-bgh2v\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.663809 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.663785    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:31.804012 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:31.803977    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:32.107138 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:32.107103    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"]
Oct 30 19:11:32.113545 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:11:32.113520    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d9ad6f_a16b_4647_a761_a417921cca9a.slice/crio-1a0921d30d8e22c769862f4ee2fd6d435c68320d6fa07a1dc6423795e22c1042 WatchSource:0}: Error finding container 1a0921d30d8e22c769862f4ee2fd6d435c68320d6fa07a1dc6423795e22c1042: Status 404 returned error can't find the container with id 1a0921d30d8e22c769862f4ee2fd6d435c68320d6fa07a1dc6423795e22c1042
Oct 30 19:11:32.115730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:32.115695    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:32.115830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:32.115770    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:32.115830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:32.115819    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:32.264706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:32.264669    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" event={"ID":"b7d9ad6f-a16b-4647-a761-a417921cca9a","Type":"ContainerStarted","Data":"e66308ee491db3d7ac854961c995ea804d1c6d94561d4d13e25652b84ba2b109"}
Oct 30 19:11:32.265254 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:32.264738    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" event={"ID":"b7d9ad6f-a16b-4647-a761-a417921cca9a","Type":"ContainerStarted","Data":"1a0921d30d8e22c769862f4ee2fd6d435c68320d6fa07a1dc6423795e22c1042"}
Oct 30 19:11:32.399767 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:32.399665    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" podStartSLOduration=1.3996489460000001 podStartE2EDuration="1.399648946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:32.398480568 +0000 UTC m=+1465.344648471" watchObservedRunningTime="2026-10-30 19:11:32.399648946 +0000 UTC m=+1465.345816850"
Oct 30 19:11:32.667107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:32.663840    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:32.667107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:32.664146    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:32.944700 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:32.944601    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:11:33.013734 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:33.013695    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:33.615056 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:11:33.598477    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:11:33.672059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:33.671183    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 19:11:33.672059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:33.671257    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 19:11:34.679046 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:34.678995    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 19:11:34.679641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:34.679607    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 19:11:35.384265 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:35.384209    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" podUID="bba0064f-500b-4920-88e7-43dc15b56e3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 19:11:35.508205 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:35.508160    5704 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" podUID="8ec41f19-292e-4376-a629-31951649bfe9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.588851 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:35.588808    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" podUID="8ec41f19-292e-4376-a629-31951649bfe9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.698846 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:35.698746    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 19:11:35.699461 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:35.699424    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 19:11:35.790056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:35.771534    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c" podUID="1d7b5569-0635-4061-b3d7-eb9890ad2184" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:35.900177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:35.874734    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8" podUID="fc21e17b-89c9-4a14-ad31-21d3ae548ee2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:36.058020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:36.057978    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl" podUID="e84096c6-01ea-407c-8d70-46f7b14bde3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:36.457118 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:36.454796    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" podUID="bba0064f-500b-4920-88e7-43dc15b56e3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:36.474060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:36.471388    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:11:36.590230 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:36.590179    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" podUID="8ec41f19-292e-4376-a629-31951649bfe9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:36.663057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:36.653854    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:11:36.698069 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:36.665757    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 19:11:36.728179 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:36.728078    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 19:11:36.941059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:36.937130    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8" podUID="fc21e17b-89c9-4a14-ad31-21d3ae548ee2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/ready\": context deadline exceeded"
Oct 30 19:11:36.962751 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:36.962698    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c" podUID="1d7b5569-0635-4061-b3d7-eb9890ad2184" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": context deadline exceeded"
Oct 30 19:11:36.996123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:36.995341    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 19:11:36.996123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:36.995405    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:11:37.090061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:37.086315    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl" podUID="e84096c6-01ea-407c-8d70-46f7b14bde3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:37.090061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:37.086835    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:37.534160 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:37.461451    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" podUID="bba0064f-500b-4920-88e7-43dc15b56e3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": context deadline exceeded"
Oct 30 19:11:37.720059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:37.712538    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" podUID="8ec41f19-292e-4376-a629-31951649bfe9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:37.794120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:37.730600    5704 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" podUID="bba0064f-500b-4920-88e7-43dc15b56e3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:37.794120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:37.731047    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 19:11:37.794120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:37.773159    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 19:11:37.959214 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:37.959164    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8" podUID="fc21e17b-89c9-4a14-ad31-21d3ae548ee2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:37.963214 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:37.963179    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c" podUID="1d7b5569-0635-4061-b3d7-eb9890ad2184" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:38.187235 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:38.187137    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl" podUID="e84096c6-01ea-407c-8d70-46f7b14bde3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:38.382691 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:38.382655    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:38.454212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:38.454006    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8"
Oct 30 19:11:38.473228 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:38.473179    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" podUID="bba0064f-500b-4920-88e7-43dc15b56e3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:38.691077 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:38.687145    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 19:11:38.740114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:38.715942    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 19:11:38.740114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:38.715881    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:38.770614 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:38.770569    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" podUID="8ec41f19-292e-4376-a629-31951649bfe9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": context deadline exceeded"
Oct 30 19:11:38.974069 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:38.966141    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c" podUID="1d7b5569-0635-4061-b3d7-eb9890ad2184" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:39.196057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:39.193495    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl" podUID="e84096c6-01ea-407c-8d70-46f7b14bde3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:39.275058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:39.274075    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:39.481115 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:39.480120    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" podUID="bba0064f-500b-4920-88e7-43dc15b56e3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:39.667492 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:39.667456    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 19:11:39.667813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:39.667766    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 19:11:39.807163 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:39.807117    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" podUID="8ec41f19-292e-4376-a629-31951649bfe9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:39.822326 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:39.822300    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:11:39.902550 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:39.902476    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:11:39.906955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:39.906923    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:11:39.993635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:39.993600    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 19:11:40.376585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:40.376190    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:11:40.376585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:40.376252    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" podUID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:11:40.664808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:40.664726    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 19:11:40.664808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:40.664774    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 19:11:41.664808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:41.664772    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused" start-of-body=
Oct 30 19:11:41.665222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:41.664837    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.70:15021/healthz/ready\": dial tcp 10.134.0.70:15021: connect: connection refused"
Oct 30 19:11:41.829736 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:41.829701    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:42.026075 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:42.026028    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:42.444726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:42.444636    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"]
Oct 30 19:11:42.492052 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:42.485211    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"
Oct 30 19:11:42.492052 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:42.485094    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"]
Oct 30 19:11:42.534702 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:42.534670    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-942c5e85\""
Oct 30 19:11:42.580396 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:42.580355    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-942c5e85-dockercfg-44ksk\""
Oct 30 19:11:42.614904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:42.614865    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c94af9-b84b-45e0-9915-e42882f53cf6-plugins-config-volume\") pod \"payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5\" (UID: \"e8c94af9-b84b-45e0-9915-e42882f53cf6\") " pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"
Oct 30 19:11:42.614904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:42.614904    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcxzr\" (UniqueName: \"kubernetes.io/projected/e8c94af9-b84b-45e0-9915-e42882f53cf6-kube-api-access-zcxzr\") pod \"payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5\" (UID: \"e8c94af9-b84b-45e0-9915-e42882f53cf6\") " pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"
Oct 30 19:11:42.716403 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:42.716314    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c94af9-b84b-45e0-9915-e42882f53cf6-plugins-config-volume\") pod \"payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5\" (UID: \"e8c94af9-b84b-45e0-9915-e42882f53cf6\") " pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"
Oct 30 19:11:42.716403 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:42.716358    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcxzr\" (UniqueName: \"kubernetes.io/projected/e8c94af9-b84b-45e0-9915-e42882f53cf6-kube-api-access-zcxzr\") pod \"payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5\" (UID: \"e8c94af9-b84b-45e0-9915-e42882f53cf6\") " pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"
Oct 30 19:11:42.717262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:42.717239    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c94af9-b84b-45e0-9915-e42882f53cf6-plugins-config-volume\") pod \"payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5\" (UID: \"e8c94af9-b84b-45e0-9915-e42882f53cf6\") " pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"
Oct 30 19:11:42.774956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:42.774923    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcxzr\" (UniqueName: \"kubernetes.io/projected/e8c94af9-b84b-45e0-9915-e42882f53cf6-kube-api-access-zcxzr\") pod \"payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5\" (UID: \"e8c94af9-b84b-45e0-9915-e42882f53cf6\") " pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"
Oct 30 19:11:42.817608 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:42.817583    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"
Oct 30 19:11:43.122385 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:43.122356    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:43.122803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:43.122774    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:43.123691 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:43.123668    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:11:43.211408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:43.211377    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"]
Oct 30 19:11:43.217858 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:11:43.217829    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8c94af9_b84b_45e0_9915_e42882f53cf6.slice/crio-9e7c1e599dd30fd0c67b83d6e542f1053b2a1e657d0991e7145d7975778318be WatchSource:0}: Error finding container 9e7c1e599dd30fd0c67b83d6e542f1053b2a1e657d0991e7145d7975778318be: Status 404 returned error can't find the container with id 9e7c1e599dd30fd0c67b83d6e542f1053b2a1e657d0991e7145d7975778318be
Oct 30 19:11:43.332104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:43.332066    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5" event={"ID":"e8c94af9-b84b-45e0-9915-e42882f53cf6","Type":"ContainerStarted","Data":"7ffa21adc09de2c7a7a440e1043418b2f4b040c27d58a1a161a6de910adda7de"}
Oct 30 19:11:43.332250 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:43.332120    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5" event={"ID":"e8c94af9-b84b-45e0-9915-e42882f53cf6","Type":"ContainerStarted","Data":"9e7c1e599dd30fd0c67b83d6e542f1053b2a1e657d0991e7145d7975778318be"}
Oct 30 19:11:43.332250 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:43.332145    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"
Oct 30 19:11:43.477211 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:43.477108    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5" podStartSLOduration=1.477094921 podStartE2EDuration="1.477094921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:43.475492162 +0000 UTC m=+1476.421660065" watchObservedRunningTime="2026-10-30 19:11:43.477094921 +0000 UTC m=+1476.423262824"
Oct 30 19:11:44.706642 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:44.706609    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:46.806005 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:46.805964    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:49.809265 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:49.809215    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:53.189896 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:53.189862    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:54.342201 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:54.342173    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"
Oct 30 19:11:54.421072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:54.420730    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:58.932259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:58.932226    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"]
Oct 30 19:11:58.936391 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:58.936373    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.018021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.017986    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.018233 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.018176    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.018316 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.018298    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.018376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.018360    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.018427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.018383    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.018465 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.018435    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.018465 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.018461    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2k86\" (UniqueName: \"kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-kube-api-access-c2k86\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.018585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.018560    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.018663 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.018648    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.036058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.036008    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"]
Oct 30 19:11:59.098680 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.098483    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:11:59.125973 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.125928    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.126235 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.126023    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.126235 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.126072    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.126235 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.126114    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.126235 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.126147    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2k86\" (UniqueName: \"kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-kube-api-access-c2k86\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.126235 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.126193    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.126705 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.126251    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.126705 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.126367    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.126705 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.126476    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.127358 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.127333    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.127672 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.127648    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.127908 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.127886    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.129510 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.129490    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.130305 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.130263    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.151146 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.151111    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.153750 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.153721    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.190889 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.190805    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.192699 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.192668    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2k86\" (UniqueName: \"kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-kube-api-access-c2k86\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.250106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.250074    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:11:59.679697 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.679665    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"]
Oct 30 19:11:59.686014 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:11:59.685990    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4687e07_e7bd_4a45_a89e_b7550ec67f95.slice/crio-a88326849351e931c1a5058bcc22bfef357428cf078f5b5928eb923597f3763a WatchSource:0}: Error finding container a88326849351e931c1a5058bcc22bfef357428cf078f5b5928eb923597f3763a: Status 404 returned error can't find the container with id a88326849351e931c1a5058bcc22bfef357428cf078f5b5928eb923597f3763a
Oct 30 19:11:59.688404 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.688370    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:59.688482 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.688436    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:11:59.688482 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:11:59.688467    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:00.466070 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.466020    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx" event={"ID":"a4687e07-e7bd-4a45-a89e-b7550ec67f95","Type":"ContainerStarted","Data":"da33beb1ece53ce055e4b336ea7dc797298a21eec23816da07ace3f83af73a30"}
Oct 30 19:12:00.466492 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.466094    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx" event={"ID":"a4687e07-e7bd-4a45-a89e-b7550ec67f95","Type":"ContainerStarted","Data":"a88326849351e931c1a5058bcc22bfef357428cf078f5b5928eb923597f3763a"}
Oct 30 19:12:00.517026 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.516996    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"]
Oct 30 19:12:00.540058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.524382    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.562969 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.562940    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"]
Oct 30 19:12:00.566946 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.566926    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.610002 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.609973    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"]
Oct 30 19:12:00.652149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.651823    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.652149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.651873    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.652149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.651949    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.652149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.652000    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.652149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.652047    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.652149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.652083    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.652669 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.652634    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dxk4\" (UniqueName: \"kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-kube-api-access-9dxk4\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.652794 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.652688    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.652794 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.652726    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.652794 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.652758    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgj4l\" (UniqueName: \"kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-kube-api-access-zgj4l\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.652794 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.652786    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.653017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.652818    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.653017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.652921    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.653017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.652953    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.653017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.652999    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29d97c69-272f-4d4d-95de-6550fdce3b3d-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.653256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.653134    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.653256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.653163    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.653256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.653245    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.657878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.657854    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"]
Oct 30 19:12:00.754542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.754506    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.754542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.754545    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.754765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.754573    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29d97c69-272f-4d4d-95de-6550fdce3b3d-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.754765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.754743    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.755022 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755001    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.755141 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.754776    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.755215 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755188    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.755270 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755250    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.755323 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755275    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.755376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755344    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.755431 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755401    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.755491 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755429    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.755491 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755468    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.755595 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755501    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dxk4\" (UniqueName: \"kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-kube-api-access-9dxk4\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.755595 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755528    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.755595 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755555    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.755595 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755566    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.755791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755629    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgj4l\" (UniqueName: \"kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-kube-api-access-zgj4l\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.755791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755657    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.755791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755666    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.755791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755718    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.755791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755731    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.755791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755737    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29d97c69-272f-4d4d-95de-6550fdce3b3d-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.756090 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755918    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.756090 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.755986    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-certs\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.756200 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.756103    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-credential-socket\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.756388 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.756347    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istiod-ca-cert\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.756388 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.756367    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-data\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.757402 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.757382    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.758281 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.758267    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.758674 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.758649    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-envoy\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.758963 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.758942    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-podinfo\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.771989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.771950    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx" podStartSLOduration=2.771938984 podStartE2EDuration="2.771938984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:00.655010213 +0000 UTC m=+1493.601178117" watchObservedRunningTime="2026-10-30 19:12:00.771938984 +0000 UTC m=+1493.718106887"
Oct 30 19:12:00.847317 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.847274    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.847507 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.847482    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-token\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.847728 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.847706    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgj4l\" (UniqueName: \"kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-kube-api-access-zgj4l\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:00.847798 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.847759    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dxk4\" (UniqueName: \"kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-kube-api-access-9dxk4\") pod \"e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") " pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.858528 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.858499    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:00.879634 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:00.879611    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:01.250502 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.250475    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:12:01.252419 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.252393    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:12:01.252548 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.252440    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx" podUID="a4687e07-e7bd-4a45-a89e-b7550ec67f95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 19:12:01.296507 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.296478    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"]
Oct 30 19:12:01.300101 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.300073    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"]
Oct 30 19:12:01.304374 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:12:01.304350    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod430edd83_d8d5_4c57_b0dc_7dfc7b98ba38.slice/crio-bd7d0e369a17eb9d2ff9b431b81c05611ee23a259ef54e96d1075316a9c09dbb WatchSource:0}: Error finding container bd7d0e369a17eb9d2ff9b431b81c05611ee23a259ef54e96d1075316a9c09dbb: Status 404 returned error can't find the container with id bd7d0e369a17eb9d2ff9b431b81c05611ee23a259ef54e96d1075316a9c09dbb
Oct 30 19:12:01.307308 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.307279    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:01.307379 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.307337    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:01.307379 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.307363    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:01.348728 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:12:01.348701    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29d97c69_272f_4d4d_95de_6550fdce3b3d.slice/crio-d81ce626ef14054226beb24f8d36c3eed11e931dd36cbae890753eb12ca79fd7 WatchSource:0}: Error finding container d81ce626ef14054226beb24f8d36c3eed11e931dd36cbae890753eb12ca79fd7: Status 404 returned error can't find the container with id d81ce626ef14054226beb24f8d36c3eed11e931dd36cbae890753eb12ca79fd7
Oct 30 19:12:01.351700 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.351663    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:01.351777 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.351734    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:01.351777 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.351768    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:01.475669 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.475641    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8" event={"ID":"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38","Type":"ContainerStarted","Data":"ec322765a72550e46a24993604b200ae4a48de875ac09569224334da7a72cbb3"}
Oct 30 19:12:01.476180 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.476159    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8" event={"ID":"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38","Type":"ContainerStarted","Data":"bd7d0e369a17eb9d2ff9b431b81c05611ee23a259ef54e96d1075316a9c09dbb"}
Oct 30 19:12:01.479567 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.479542    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk" event={"ID":"29d97c69-272f-4d4d-95de-6550fdce3b3d","Type":"ContainerStarted","Data":"cf509bbb717d0dd6bea801422bc9982cba0c554e9e7627a64ff56e112c370ea8"}
Oct 30 19:12:01.479680 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.479591    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk" event={"ID":"29d97c69-272f-4d4d-95de-6550fdce3b3d","Type":"ContainerStarted","Data":"d81ce626ef14054226beb24f8d36c3eed11e931dd36cbae890753eb12ca79fd7"}
Oct 30 19:12:01.595182 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.595081    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8" podStartSLOduration=1.595067105 podStartE2EDuration="1.595067105s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:01.594071294 +0000 UTC m=+1494.540239199" watchObservedRunningTime="2026-10-30 19:12:01.595067105 +0000 UTC m=+1494.541235060"
Oct 30 19:12:01.727067 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.726872    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk" podStartSLOduration=1.726853972 podStartE2EDuration="1.726853972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:01.717353378 +0000 UTC m=+1494.663521283" watchObservedRunningTime="2026-10-30 19:12:01.726853972 +0000 UTC m=+1494.673021878"
Oct 30 19:12:01.859709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.859609    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:01.880285 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:01.880252    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:02.065057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:02.064241    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:02.263192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:02.263137    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:12:02.263383 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:02.263200    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx" podUID="a4687e07-e7bd-4a45-a89e-b7550ec67f95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 19:12:02.559208 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:02.554924    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:02.845108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:02.844436    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:02.981057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:02.962902    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 19:12:02.981057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:02.962967    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8" podUID="430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 19:12:02.981057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:02.965014    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:12:02.981057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:02.965076    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk" podUID="29d97c69-272f-4d4d-95de-6550fdce3b3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:12:03.262585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:03.262150    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:12:03.262585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:03.262208    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx" podUID="a4687e07-e7bd-4a45-a89e-b7550ec67f95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 19:12:03.314069 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:03.309360    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:03.529113 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:12:03.525249    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:12:03.860112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:03.859584    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 19:12:03.860112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:03.859658    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8" podUID="430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 19:12:03.881454 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:03.881417    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:12:03.881619 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:03.881479    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk" podUID="29d97c69-272f-4d4d-95de-6550fdce3b3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:12:03.911527 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:03.911494    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:04.188115 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:04.186917    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:04.269056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:04.258438    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:12:04.269056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:04.258497    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx" podUID="a4687e07-e7bd-4a45-a89e-b7550ec67f95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 19:12:04.860746 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:04.860401    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 19:12:04.860746 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:04.860464    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8" podUID="430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 19:12:04.881554 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:04.881512    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:12:04.881726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:04.881576    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk" podUID="29d97c69-272f-4d4d-95de-6550fdce3b3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:12:05.253475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:05.253441    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:12:05.253758 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:05.253728    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx" podUID="a4687e07-e7bd-4a45-a89e-b7550ec67f95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 19:12:05.863461 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:05.863424    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 19:12:05.864443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:05.864411    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8" podUID="430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 19:12:05.882427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:05.882392    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:12:05.882673 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:05.882647    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk" podUID="29d97c69-272f-4d4d-95de-6550fdce3b3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:12:06.263057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:06.257358    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused" start-of-body=
Oct 30 19:12:06.263057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:06.257415    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx" podUID="a4687e07-e7bd-4a45-a89e-b7550ec67f95" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.72:15021/healthz/ready\": dial tcp 10.134.0.72:15021: connect: connection refused"
Oct 30 19:12:06.870059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:06.859382    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 19:12:06.870059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:06.859445    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8" podUID="430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 19:12:06.880710 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:06.880576    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:12:06.880710 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:06.880640    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk" podUID="29d97c69-272f-4d4d-95de-6550fdce3b3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:12:07.869129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:07.869095    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 19:12:07.869417 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:07.869388    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8" podUID="430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 19:12:07.884059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:07.883383    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:12:07.884059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:07.883421    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk" podUID="29d97c69-272f-4d4d-95de-6550fdce3b3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:12:07.977692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:07.977655    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:12:07.978249 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:07.978223    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:12:07.979143 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:07.979120    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:12:08.412497 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:08.412457    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:08.619955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:08.619892    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:08.859663 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:08.859628    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused" start-of-body=
Oct 30 19:12:08.859865 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:08.859689    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8" podUID="430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.73:15021/healthz/ready\": dial tcp 10.134.0.73:15021: connect: connection refused"
Oct 30 19:12:08.880050 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:08.880001    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused" start-of-body=
Oct 30 19:12:08.880263 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:08.880081    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk" podUID="29d97c69-272f-4d4d-95de-6550fdce3b3d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.74:15021/healthz/ready\": dial tcp 10.134.0.74:15021: connect: connection refused"
Oct 30 19:12:09.192641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:09.192543    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:09.393536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:09.393498    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:09.582504 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:09.582460    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:09.863587 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:09.863501    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:09.863900 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:09.863878    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:09.864597 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:09.864578    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:12:09.884596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:09.884574    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:09.884850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:09.884833    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:09.885395 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:09.885378    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:12:12.614166 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:12.613223    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:12.822076 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:12.821981    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:13.040469 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:13.040433    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"]
Oct 30 19:12:13.063419 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:13.063390    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"
Oct 30 19:12:13.135051 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:13.135007    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"]
Oct 30 19:12:13.174086 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:13.172271    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d696b299-3a8b-49a0-be2f-127c6967ecc0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz\" (UID: \"d696b299-3a8b-49a0-be2f-127c6967ecc0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"
Oct 30 19:12:13.174086 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:13.172456    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbqrq\" (UniqueName: \"kubernetes.io/projected/d696b299-3a8b-49a0-be2f-127c6967ecc0-kube-api-access-dbqrq\") pod \"payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz\" (UID: \"d696b299-3a8b-49a0-be2f-127c6967ecc0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"
Oct 30 19:12:13.222054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:13.221818    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-4f4614-dockercfg-qghhz\""
Oct 30 19:12:13.222054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:13.221915    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-4f4614\""
Oct 30 19:12:13.308102 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:13.293802    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d696b299-3a8b-49a0-be2f-127c6967ecc0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz\" (UID: \"d696b299-3a8b-49a0-be2f-127c6967ecc0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"
Oct 30 19:12:13.371088 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:13.371051    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbqrq\" (UniqueName: \"kubernetes.io/projected/d696b299-3a8b-49a0-be2f-127c6967ecc0-kube-api-access-dbqrq\") pod \"payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz\" (UID: \"d696b299-3a8b-49a0-be2f-127c6967ecc0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"
Oct 30 19:12:13.387198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:13.299969    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d696b299-3a8b-49a0-be2f-127c6967ecc0-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz\" (UID: \"d696b299-3a8b-49a0-be2f-127c6967ecc0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"
Oct 30 19:12:13.942142 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:13.929305    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbqrq\" (UniqueName: \"kubernetes.io/projected/d696b299-3a8b-49a0-be2f-127c6967ecc0-kube-api-access-dbqrq\") pod \"payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz\" (UID: \"d696b299-3a8b-49a0-be2f-127c6967ecc0\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"
Oct 30 19:12:13.989066 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:13.988290    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"]
Oct 30 19:12:14.005665 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:14.005425    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"
Oct 30 19:12:14.018631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:14.016206    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"]
Oct 30 19:12:14.018631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:14.016316    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"
Oct 30 19:12:14.093066 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:14.091880    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20503cda-6f8a-4b59-8027-462e65a4a5ef-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q\" (UID: \"20503cda-6f8a-4b59-8027-462e65a4a5ef\") " pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"
Oct 30 19:12:14.093066 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:14.091922    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqf8q\" (UniqueName: \"kubernetes.io/projected/20503cda-6f8a-4b59-8027-462e65a4a5ef-kube-api-access-tqf8q\") pod \"payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q\" (UID: \"20503cda-6f8a-4b59-8027-462e65a4a5ef\") " pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"
Oct 30 19:12:14.198107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:14.197909    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20503cda-6f8a-4b59-8027-462e65a4a5ef-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q\" (UID: \"20503cda-6f8a-4b59-8027-462e65a4a5ef\") " pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"
Oct 30 19:12:14.199084 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:14.199058    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20503cda-6f8a-4b59-8027-462e65a4a5ef-plugins-config-volume\") pod \"payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q\" (UID: \"20503cda-6f8a-4b59-8027-462e65a4a5ef\") " pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"
Oct 30 19:12:14.199343 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:14.199325    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqf8q\" (UniqueName: \"kubernetes.io/projected/20503cda-6f8a-4b59-8027-462e65a4a5ef-kube-api-access-tqf8q\") pod \"payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q\" (UID: \"20503cda-6f8a-4b59-8027-462e65a4a5ef\") " pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"
Oct 30 19:12:14.306014 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:14.305975    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqf8q\" (UniqueName: \"kubernetes.io/projected/20503cda-6f8a-4b59-8027-462e65a4a5ef-kube-api-access-tqf8q\") pod \"payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q\" (UID: \"20503cda-6f8a-4b59-8027-462e65a4a5ef\") " pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"
Oct 30 19:12:14.463675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:14.463328    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"
Oct 30 19:12:14.949224 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:14.949194    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"]
Oct 30 19:12:15.063122 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:15.048777    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"]
Oct 30 19:12:15.678421 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:15.678383    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:16.019097 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:16.017394    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q" event={"ID":"20503cda-6f8a-4b59-8027-462e65a4a5ef","Type":"ContainerStarted","Data":"be58e10e31b65ff6f66dc10bdede9a67c1ebcd6b8f2acf11202fd7788567f8f3"}
Oct 30 19:12:16.099057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:16.093632    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz" event={"ID":"d696b299-3a8b-49a0-be2f-127c6967ecc0","Type":"ContainerStarted","Data":"540407c373260223f9dee312723d3b903e0adb5431b202b9f381e6fe0b16c8e6"}
Oct 30 19:12:16.099057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:16.093697    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz" event={"ID":"d696b299-3a8b-49a0-be2f-127c6967ecc0","Type":"ContainerStarted","Data":"bb11239d1b5ee52d4c6833801e0627162adc1d1b2bdd415c32f8934fe5288cb2"}
Oct 30 19:12:16.099057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:16.094700    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"
Oct 30 19:12:16.234116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:16.232661    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz" podStartSLOduration=4.232641643 podStartE2EDuration="4.232641643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:16.230731798 +0000 UTC m=+1509.176899726" watchObservedRunningTime="2026-10-30 19:12:16.232641643 +0000 UTC m=+1509.178809549"
Oct 30 19:12:17.139093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:17.139056    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q" event={"ID":"20503cda-6f8a-4b59-8027-462e65a4a5ef","Type":"ContainerStarted","Data":"916588c3a7a03eba98ff1c9a8086200923ca4b37c99f21427488a2119aeceb99"}
Oct 30 19:12:17.139771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:17.139748    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"
Oct 30 19:12:17.289522 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:17.289412    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q" podStartSLOduration=4.289392283 podStartE2EDuration="4.289392283s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:17.280820274 +0000 UTC m=+1510.226988171" watchObservedRunningTime="2026-10-30 19:12:17.289392283 +0000 UTC m=+1510.235560187"
Oct 30 19:12:20.644173 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:20.644102    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:25.275767 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:25.275730    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:28.147312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:28.147281    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"
Oct 30 19:12:28.147784 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:28.147350    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"
Oct 30 19:12:29.209127 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.209095    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"]
Oct 30 19:12:29.213310 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.213288    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.276841 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.276812    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8t8w\" (UniqueName: \"kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-kube-api-access-s8t8w\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.277090 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.277069    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.277189 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.277135    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fda20ce9-bc4f-416d-8943-7fefd92f810a-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.277300 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.277278    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.277429 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.277312    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.277429 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.277374    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.277429 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.277394    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.277601 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.277468    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.277601 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.277492    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.293686 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.293661    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"]
Oct 30 19:12:29.378659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.378628    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8t8w\" (UniqueName: \"kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-kube-api-access-s8t8w\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.378833 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.378689    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.378833 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.378744    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fda20ce9-bc4f-416d-8943-7fefd92f810a-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.378957 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.378840    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.378957 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.378877    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.378957 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.378917    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.378957 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.378941    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.379212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.378999    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.379212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.379047    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.379212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.379076    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-data\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.379384 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.379365    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-credential-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.379443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.379412    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-certs\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.379509 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.379475    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-socket\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.379560 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.379521    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fda20ce9-bc4f-416d-8943-7fefd92f810a-istiod-ca-cert\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.381220 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.381198    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-envoy\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.381374 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.381356    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-podinfo\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.440756 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.440714    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-token\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.447720 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.447692    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8t8w\" (UniqueName: \"kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-kube-api-access-s8t8w\") pod \"e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") " pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.526824 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.526788    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:29.960071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.959980    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"]
Oct 30 19:12:29.967733 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:12:29.967706    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfda20ce9_bc4f_416d_8943_7fefd92f810a.slice/crio-b00919dbe518df94837ebdc91dec22a1ec72e03b7150b1dece69d1e93cce96f5 WatchSource:0}: Error finding container b00919dbe518df94837ebdc91dec22a1ec72e03b7150b1dece69d1e93cce96f5: Status 404 returned error can't find the container with id b00919dbe518df94837ebdc91dec22a1ec72e03b7150b1dece69d1e93cce96f5
Oct 30 19:12:29.970294 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.970263    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:29.970376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.970327    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:29.970376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:29.970361    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:12:30.039063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:30.037626    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:30.232514 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:30.232419    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd" event={"ID":"fda20ce9-bc4f-416d-8943-7fefd92f810a","Type":"ContainerStarted","Data":"9f12360dc808459b64c9137ab3d51a336229b1ac57042fbe70bf677debc43e45"}
Oct 30 19:12:30.232967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:30.232525    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd" event={"ID":"fda20ce9-bc4f-416d-8943-7fefd92f810a","Type":"ContainerStarted","Data":"b00919dbe518df94837ebdc91dec22a1ec72e03b7150b1dece69d1e93cce96f5"}
Oct 30 19:12:30.348493 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:30.348440    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd" podStartSLOduration=1.348427414 podStartE2EDuration="1.348427414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:30.346973875 +0000 UTC m=+1523.293141803" watchObservedRunningTime="2026-10-30 19:12:30.348427414 +0000 UTC m=+1523.294595316"
Oct 30 19:12:30.529058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:30.527900    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:31.544173 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:31.544140    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:12:31.544658 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:31.544199    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd" podUID="fda20ce9-bc4f-416d-8943-7fefd92f810a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 19:12:32.540057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:32.528360    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:12:32.540057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:32.528418    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd" podUID="fda20ce9-bc4f-416d-8943-7fefd92f810a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 19:12:33.383072 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:12:33.383025    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:12:33.543056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:33.542410    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:12:33.543056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:33.542479    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd" podUID="fda20ce9-bc4f-416d-8943-7fefd92f810a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 19:12:34.538429 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:34.538393    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:12:34.539078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:34.539018    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd" podUID="fda20ce9-bc4f-416d-8943-7fefd92f810a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 19:12:34.842108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:34.836082    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:35.533062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:35.527370    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:12:35.533062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:35.527430    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd" podUID="fda20ce9-bc4f-416d-8943-7fefd92f810a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 19:12:36.527878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:36.527846    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused" start-of-body=
Oct 30 19:12:36.528397 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:36.527893    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd" podUID="fda20ce9-bc4f-416d-8943-7fefd92f810a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.77:15021/healthz/ready\": dial tcp 10.134.0.77:15021: connect: connection refused"
Oct 30 19:12:37.531661 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:37.531626    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:37.532047 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:37.531920    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:37.532505 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:37.532490    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:12:39.487102 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:39.487068    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:43.808675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:43.808645    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"]
Oct 30 19:12:43.814275 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:43.814253    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:43.863536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:43.863515    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-4f4614-serving-cert\""
Oct 30 19:12:43.911892 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:43.911863    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"]
Oct 30 19:12:43.995959 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:43.995929    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-metrics-certs\") pod \"maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:43.996131 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:43.995981    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-maas-api-tls\") pod \"maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:43.996131 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:43.996090    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-openshift-service-ca\") pod \"maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:43.996214 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:43.996132    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wlvr\" (UniqueName: \"kubernetes.io/projected/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-kube-api-access-5wlvr\") pod \"maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:44.030921 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.030889    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:44.104179 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.098621    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-metrics-certs\") pod \"maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:44.104179 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.098985    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-maas-api-tls\") pod \"maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:44.104179 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.099162    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-openshift-service-ca\") pod \"maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:44.104179 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.099316    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wlvr\" (UniqueName: \"kubernetes.io/projected/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-kube-api-access-5wlvr\") pod \"maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:44.104179 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:12:44.099957    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-serving-cert: secret "maas-api-e2e-api-a-4f4614-serving-cert" not found
Oct 30 19:12:44.104179 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:12:44.100066    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-maas-api-tls podName:03d301d0-7d5a-42e3-9e91-b63a3c1520fd nodeName:}" failed. No retries permitted until 2026-10-30 19:12:44.600026772 +0000 UTC m=+1537.546194671 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-maas-api-tls") pod "maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl" (UID: "03d301d0-7d5a-42e3-9e91-b63a3c1520fd") : secret "maas-api-e2e-api-a-4f4614-serving-cert" not found
Oct 30 19:12:44.104179 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.100782    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-openshift-service-ca\") pod \"maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:44.128779 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.128748    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-metrics-certs\") pod \"maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:44.159582 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.159551    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wlvr\" (UniqueName: \"kubernetes.io/projected/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-kube-api-access-5wlvr\") pod \"maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:44.253888 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.253849    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:44.455743 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.455649    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:44.503538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.503506    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:12:44.610398 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.610362    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-maas-api-tls\") pod \"maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:44.612976 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.612949    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-maas-api-tls\") pod \"maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:44.729859 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:44.729771    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:45.151861 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:45.151841    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"]
Oct 30 19:12:45.154107 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:12:45.154021    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03d301d0_7d5a_42e3_9e91_b63a3c1520fd.slice/crio-e7b7002e660fb7da0dd2e952413a04cec571147b04bc912ec321e95cca22325f WatchSource:0}: Error finding container e7b7002e660fb7da0dd2e952413a04cec571147b04bc912ec321e95cca22325f: Status 404 returned error can't find the container with id e7b7002e660fb7da0dd2e952413a04cec571147b04bc912ec321e95cca22325f
Oct 30 19:12:45.155614 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:45.155594    5704 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:12:45.449014 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:45.448923    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl" event={"ID":"03d301d0-7d5a-42e3-9e91-b63a3c1520fd","Type":"ContainerStarted","Data":"e7b7002e660fb7da0dd2e952413a04cec571147b04bc912ec321e95cca22325f"}
Oct 30 19:12:46.463113 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:46.461964    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl" event={"ID":"03d301d0-7d5a-42e3-9e91-b63a3c1520fd","Type":"ContainerStarted","Data":"8489d8f9792c1e9930ddeeed77d87ee1953c6df1a48faa3aa9577ac952cda3c1"}
Oct 30 19:12:46.463113 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:46.462986    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:48.235579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:48.234314    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl" podUID="e84096c6-01ea-407c-8d70-46f7b14bde3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": context deadline exceeded"
Oct 30 19:12:48.547170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:48.547119    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8" podUID="fc21e17b-89c9-4a14-ad31-21d3ae548ee2" containerName="main" probeResult="failure" output="Get \"https://10.134.0.35:8000/ready\": context deadline exceeded"
Oct 30 19:12:48.548537 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:48.548506    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" podUID="bba0064f-500b-4920-88e7-43dc15b56e3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:48.786056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:48.773773    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" podUID="8ec41f19-292e-4376-a629-31951649bfe9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:49.067210 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:49.067166    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c" podUID="1d7b5569-0635-4061-b3d7-eb9890ad2184" containerName="main" probeResult="failure" output="Get \"https://10.134.0.55:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:49.251512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:49.251464    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:49.254673 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:49.254641    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl" podUID="e84096c6-01ea-407c-8d70-46f7b14bde3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:12:49.285479 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:49.285439    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:12:49.394974 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:49.394890    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:12:49.401011 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:49.399334    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl" podStartSLOduration=5.421550607 podStartE2EDuration="6.399314466s" totalImagesPullingTime="977.763859ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:46.578301685 +0000 UTC m=+1539.524469588" watchObservedRunningTime="2026-10-30 19:12:49.399314466 +0000 UTC m=+1542.345482370"
Oct 30 19:12:49.864146 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:49.863147    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:50.104128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:50.101906    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:53.724991 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:53.724964    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:12:53.865899 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:53.865864    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:54.121056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:54.117690    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:55.041021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:55.040984    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:58.680917 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:58.680866    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:58.917663 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:58.917630    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:59.094279 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.094247    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"]
Oct 30 19:12:59.098139 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.098113    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"
Oct 30 19:12:59.133984 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.133951    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:12:59.150551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.150532    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-c4ca9a\""
Oct 30 19:12:59.274061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.241186    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-c4ca9a-dockercfg-27glf\""
Oct 30 19:12:59.274392 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.274363    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"]
Oct 30 19:12:59.390110 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.389573    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6a1e289-e8f5-456f-8d7c-1eca2944324b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn\" (UID: \"c6a1e289-e8f5-456f-8d7c-1eca2944324b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"
Oct 30 19:12:59.409076 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.391637    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqgfp\" (UniqueName: \"kubernetes.io/projected/c6a1e289-e8f5-456f-8d7c-1eca2944324b-kube-api-access-lqgfp\") pod \"payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn\" (UID: \"c6a1e289-e8f5-456f-8d7c-1eca2944324b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"
Oct 30 19:12:59.492529 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.492497    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6a1e289-e8f5-456f-8d7c-1eca2944324b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn\" (UID: \"c6a1e289-e8f5-456f-8d7c-1eca2944324b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"
Oct 30 19:12:59.492696 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.492632    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqgfp\" (UniqueName: \"kubernetes.io/projected/c6a1e289-e8f5-456f-8d7c-1eca2944324b-kube-api-access-lqgfp\") pod \"payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn\" (UID: \"c6a1e289-e8f5-456f-8d7c-1eca2944324b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"
Oct 30 19:12:59.493310 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.493290    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6a1e289-e8f5-456f-8d7c-1eca2944324b-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn\" (UID: \"c6a1e289-e8f5-456f-8d7c-1eca2944324b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"
Oct 30 19:12:59.552083 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.552058    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqgfp\" (UniqueName: \"kubernetes.io/projected/c6a1e289-e8f5-456f-8d7c-1eca2944324b-kube-api-access-lqgfp\") pod \"payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn\" (UID: \"c6a1e289-e8f5-456f-8d7c-1eca2944324b\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"
Oct 30 19:12:59.711575 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.711499    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"
Oct 30 19:12:59.913551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.913518    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"]
Oct 30 19:12:59.917718 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.917696    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:12:59.959141 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.959112    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"]
Oct 30 19:12:59.962247 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.962205    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:12:59.999989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:12:59.999959    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-kube-api-access-fpp68\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.000156 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.000112    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.000226 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.000168    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.000303 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.000287    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.000373 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.000320    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.000489 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.000460    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.000551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.000526    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.000656 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.000629    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.000737 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.000722    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.018540 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.018516    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"]
Oct 30 19:13:00.065937 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.065909    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"]
Oct 30 19:13:00.094961 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.094931    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"]
Oct 30 19:13:00.101771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.101742    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.101894 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.101818    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.101965 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.101903    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.101965 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.101946    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.102097 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.102007    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.102097 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.102070    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.102408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.102387    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.102525 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.102505    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.102605 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.102545    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-kube-api-access-fpp68\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.102670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.102604    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.102670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.102608    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.102775 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.102669    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl28x\" (UniqueName: \"kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-kube-api-access-cl28x\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.102775 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.102706    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f78f6217-90d4-405c-bed9-36b9e9911b7c-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.102775 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.102744    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.102929 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.102790    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.102929 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.102821    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.103474 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.103151    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.103778 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.103748    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.103871 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.103796    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.103871 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.103835    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.104606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.104291    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.104606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.104403    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.107753 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.107735    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.108557 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.108503    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.110354 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.110328    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.160954 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.160932    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.161203 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.161182    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-kube-api-access-fpp68\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.206005 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.205979    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.206135 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.206088    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.206249 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.206235    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.206297 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.206258    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.206336 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.206292    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.206367 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.206338    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.206367 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.206359    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl28x\" (UniqueName: \"kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-kube-api-access-cl28x\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.206454 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.206391    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f78f6217-90d4-405c-bed9-36b9e9911b7c-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.206454 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.206440    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.206559 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.206460    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.206559 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.206530    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.206695 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.206662    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.206767 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.206747    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.207147 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.207130    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f78f6217-90d4-405c-bed9-36b9e9911b7c-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.208451 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.208433    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.208541 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.208524    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.233062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.232990    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:00.269293 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.269260    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl28x\" (UniqueName: \"kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-kube-api-access-cl28x\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.269618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.269595    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.274413 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.274230    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:00.349955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.349921    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:00.673272 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.673240    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"]
Oct 30 19:13:00.679506 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:13:00.679479    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb6977e8_1680_449b_ad3e_3434eac1e0e9.slice/crio-7716e6ee6624575d1f78a5ee019d82efe0e5305f281d691f19b75343121ce48c WatchSource:0}: Error finding container 7716e6ee6624575d1f78a5ee019d82efe0e5305f281d691f19b75343121ce48c: Status 404 returned error can't find the container with id 7716e6ee6624575d1f78a5ee019d82efe0e5305f281d691f19b75343121ce48c
Oct 30 19:13:00.682563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.682532    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:13:00.682670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.682612    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:13:00.682670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.682663    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:13:00.772247 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.772216    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"]
Oct 30 19:13:00.794065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.793997    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:13:00.794169 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.794121    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:13:00.794226 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.794180    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:13:00.940457 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.940364    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn" event={"ID":"c6a1e289-e8f5-456f-8d7c-1eca2944324b","Type":"ContainerStarted","Data":"ff700314c82fefa48403cf88cfb58c767d0602f7d688b5f1f3e97a2eb7adf740"}
Oct 30 19:13:00.940457 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.940434    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn" event={"ID":"c6a1e289-e8f5-456f-8d7c-1eca2944324b","Type":"ContainerStarted","Data":"43c5e69e6a0e1ee36e7f85c41ca992211d61f077704a559f101c65bc576aebce"}
Oct 30 19:13:00.942544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.942495    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" event={"ID":"f78f6217-90d4-405c-bed9-36b9e9911b7c","Type":"ContainerStarted","Data":"019fdd8c5e1d451d6489b46540b16af971713bb0002d517724577dabcc3b5622"}
Oct 30 19:13:00.942673 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.942573    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" event={"ID":"f78f6217-90d4-405c-bed9-36b9e9911b7c","Type":"ContainerStarted","Data":"5f28919f1d85606ef12d876df30addf517031007c943caa3b5d224cd28f46e37"}
Oct 30 19:13:00.944554 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.944524    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" event={"ID":"eb6977e8-1680-449b-ad3e-3434eac1e0e9","Type":"ContainerStarted","Data":"82f89b20be049ce7a1db619a6dc4f0281895300c03aa97766acaa4bc34c62ab9"}
Oct 30 19:13:00.944659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:00.944585    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" event={"ID":"eb6977e8-1680-449b-ad3e-3434eac1e0e9","Type":"ContainerStarted","Data":"7716e6ee6624575d1f78a5ee019d82efe0e5305f281d691f19b75343121ce48c"}
Oct 30 19:13:01.054383 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:01.054331    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn" podStartSLOduration=2.054316633 podStartE2EDuration="2.054316633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:01.053000216 +0000 UTC m=+1553.999168119" watchObservedRunningTime="2026-10-30 19:13:01.054316633 +0000 UTC m=+1554.000484575"
Oct 30 19:13:01.170446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:01.170385    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" podStartSLOduration=2.170366933 podStartE2EDuration="2.170366933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:01.169103633 +0000 UTC m=+1554.115271542" watchObservedRunningTime="2026-10-30 19:13:01.170366933 +0000 UTC m=+1554.116534839"
Oct 30 19:13:01.233854 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:01.233779    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:01.235097 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:01.235074    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:13:01.235204 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:01.235120    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 19:13:01.275004 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:01.274980    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:01.276621 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:01.276598    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:13:01.276742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:01.276647    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" podUID="f78f6217-90d4-405c-bed9-36b9e9911b7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 19:13:01.287896 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:01.287841    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" podStartSLOduration=2.287824518 podStartE2EDuration="2.287824518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:01.282580742 +0000 UTC m=+1554.228748644" watchObservedRunningTime="2026-10-30 19:13:01.287824518 +0000 UTC m=+1554.233992424"
Oct 30 19:13:01.974688 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:01.974654    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"
Oct 30 19:13:02.257114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:02.249721    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:13:02.257114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:02.249782    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 19:13:02.285208 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:02.285172    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:13:02.285479 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:02.285453    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" podUID="f78f6217-90d4-405c-bed9-36b9e9911b7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 19:13:02.947058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:02.946064    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:03.242115 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:03.235165    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:13:03.242115 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:03.235233    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 19:13:03.281085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:03.280839    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:13:03.281085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:03.280922    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" podUID="f78f6217-90d4-405c-bed9-36b9e9911b7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 19:13:03.402054 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:13:03.401837    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:13:04.244059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:04.240479    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:13:04.244059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:04.240546    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 19:13:04.282057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:04.276421    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:13:04.282057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:04.276494    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" podUID="f78f6217-90d4-405c-bed9-36b9e9911b7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 19:13:05.237153 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:05.234519    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:13:05.237153 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:05.234596    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 19:13:05.275440 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:05.275382    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:13:05.275830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:05.275442    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" podUID="f78f6217-90d4-405c-bed9-36b9e9911b7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 19:13:05.463023 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:05.462986    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:06.236453 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:06.236417    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:13:06.236773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:06.236734    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 19:13:06.275186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:06.275158    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:13:06.275390 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:06.275370    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" podUID="f78f6217-90d4-405c-bed9-36b9e9911b7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 19:13:07.240882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:07.240411    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:13:07.240882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:07.240470    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 19:13:07.275131 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:07.275094    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:13:07.275297 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:07.275157    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" podUID="f78f6217-90d4-405c-bed9-36b9e9911b7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 19:13:07.674542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:07.674508    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:08.233891 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:08.233856    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused" start-of-body=
Oct 30 19:13:08.234094 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:08.233919    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": dial tcp 10.134.0.80:15021: connect: connection refused"
Oct 30 19:13:08.275690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:08.275653    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Oct 30 19:13:08.276115 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:08.275713    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" podUID="f78f6217-90d4-405c-bed9-36b9e9911b7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Oct 30 19:13:09.416829 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:09.416777    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:09.417704 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:09.417681    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:09.418454 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:09.418437    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:13:10.276324 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:10.276270    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:13:10.276658 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:10.276628    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.80:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:13:10.363689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:10.363646    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:10.554495 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:10.554409    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:10.556608 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:10.556590    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:10.558057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:10.558013    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:13:13.003582 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:13.003556    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"
Oct 30 19:13:15.081923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:15.081633    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:17.603408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:17.603375    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:18.989282 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:18.989250    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:19.804194 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:19.804142    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:24.371264 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:24.371230    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:27.367693 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:27.367658    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:13:28.375249 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:28.375218    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:29.559057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:29.552855    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:33.385051 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:13:33.385006    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:13:34.209059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.208999    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"]
Oct 30 19:13:34.214395 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.214375    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:34.273361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.273337    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-c4ca9a-serving-cert\""
Oct 30 19:13:34.302212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.302184    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-metrics-certs\") pod \"maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:34.302212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.302222    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-openshift-service-ca\") pod \"maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:34.302440 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.302353    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktzgc\" (UniqueName: \"kubernetes.io/projected/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-kube-api-access-ktzgc\") pod \"maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:34.302764 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.302737    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-maas-api-tls\") pod \"maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:34.321297 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.321275    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"]
Oct 30 19:13:34.403671 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.403645    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-maas-api-tls\") pod \"maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:34.404006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.403689    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-metrics-certs\") pod \"maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:34.404006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.403712    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-openshift-service-ca\") pod \"maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:34.404006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.403750    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktzgc\" (UniqueName: \"kubernetes.io/projected/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-kube-api-access-ktzgc\") pod \"maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:34.404006 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:13:34.403799    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-serving-cert: secret "maas-api-e2e-api-a-c4ca9a-serving-cert" not found
Oct 30 19:13:34.404006 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:13:34.403877    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-maas-api-tls podName:bbba3e0b-6924-4b8a-bb5c-47e40fa39c59 nodeName:}" failed. No retries permitted until 2026-10-30 19:13:34.903855549 +0000 UTC m=+1587.850023447 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-maas-api-tls") pod "maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7" (UID: "bbba3e0b-6924-4b8a-bb5c-47e40fa39c59") : secret "maas-api-e2e-api-a-c4ca9a-serving-cert" not found
Oct 30 19:13:34.404445 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.404429    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-openshift-service-ca\") pod \"maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:34.406107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.406081    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-metrics-certs\") pod \"maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:34.484821 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.476847    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktzgc\" (UniqueName: \"kubernetes.io/projected/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-kube-api-access-ktzgc\") pod \"maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:34.485714 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.485591    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:34.694102 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.693224    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:34.866380 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.866342    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:34.921060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.917379    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-maas-api-tls\") pod \"maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:34.944291 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:34.941814    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-maas-api-tls\") pod \"maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:35.050977 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:35.050937    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:35.127809 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:35.127725    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:35.509766 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:35.509736    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"]
Oct 30 19:13:35.516175 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:13:35.516149    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbba3e0b_6924_4b8a_bb5c_47e40fa39c59.slice/crio-62336a8d94af2cfcef46816ab678f1f51b6acf7135741d5371edbbdb880a026f WatchSource:0}: Error finding container 62336a8d94af2cfcef46816ab678f1f51b6acf7135741d5371edbbdb880a026f: Status 404 returned error can't find the container with id 62336a8d94af2cfcef46816ab678f1f51b6acf7135741d5371edbbdb880a026f
Oct 30 19:13:35.786877 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:35.786789    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:36.296926 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:36.296885    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7" event={"ID":"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59","Type":"ContainerStarted","Data":"62336a8d94af2cfcef46816ab678f1f51b6acf7135741d5371edbbdb880a026f"}
Oct 30 19:13:37.341091 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:37.341007    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7" event={"ID":"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59","Type":"ContainerStarted","Data":"b178a5f1ed41425cfd42fdef367e3217029d4fd7347c2e42bf5f7098bd6ceb85"}
Oct 30 19:13:37.346385 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:37.342258    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:37.454918 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:37.454865    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7" podStartSLOduration=2.4586943039999998 podStartE2EDuration="3.454849328s" totalImagesPullingTime="996.155024ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:13:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:37.453042683 +0000 UTC m=+1590.399210579" watchObservedRunningTime="2026-10-30 19:13:37.454849328 +0000 UTC m=+1590.401017232"
Oct 30 19:13:39.370124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:39.370081    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:44.024021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:44.020507    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:44.360192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:44.360111    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:13:45.749328 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:45.749291    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:46.647057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:46.612516    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:47.209132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:47.209097    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:48.527058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.505809    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:48.712046 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.711988    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"]
Oct 30 19:13:48.717827 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.717802    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.731808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.731780    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:48.770644 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.770616    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-942c5e85-openshift-default-dockercfg-2q6j6\""
Oct 30 19:13:48.821109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.818165    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.821109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.818220    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbb431a4-89ed-438b-9f24-5573b880f19c-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.821109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.818269    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.821109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.818420    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.821109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.818469    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.821109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.818526    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.821109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.818548    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.821109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.818593    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thpjq\" (UniqueName: \"kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-kube-api-access-thpjq\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.821109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.818623    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.821109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.821018    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"]
Oct 30 19:13:48.919907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.919879    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.920093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.919918    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbb431a4-89ed-438b-9f24-5573b880f19c-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.920093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.920057    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.920189 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.920155    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.920242 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.920185    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.920308 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.920273    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.920363 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.920307    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.920426 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.920369    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thpjq\" (UniqueName: \"kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-kube-api-access-thpjq\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.920426 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.920409    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.920426 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.920417    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.920634 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.920616    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbb431a4-89ed-438b-9f24-5573b880f19c-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.920843 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.920824    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.921027 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.921011    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.921150 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.921131    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.923058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.923025    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:48.924455 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.924437    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:49.000007 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:48.999913    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:49.030379 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:49.030341    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thpjq\" (UniqueName: \"kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-kube-api-access-thpjq\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:49.033795 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:49.033733    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:49.036530 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:49.036327    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:49.503188 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:49.503160    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"]
Oct 30 19:13:49.508574 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:13:49.508550    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbb431a4_89ed_438b_9f24_5573b880f19c.slice/crio-2a630408f894d26cb431473ae6221aab71560db2f5691127cf577f38d94faf53 WatchSource:0}: Error finding container 2a630408f894d26cb431473ae6221aab71560db2f5691127cf577f38d94faf53: Status 404 returned error can't find the container with id 2a630408f894d26cb431473ae6221aab71560db2f5691127cf577f38d94faf53
Oct 30 19:13:49.510956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:49.510923    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:13:49.511024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:49.510989    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:13:49.511024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:49.511018    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:13:49.948607 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:49.948534    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk" event={"ID":"dbb431a4-89ed-438b-9f24-5573b880f19c","Type":"ContainerStarted","Data":"be1e80316a6db2509d19a6586ca4b2154be59234fe75094b385d995cbd394dc7"}
Oct 30 19:13:49.948607 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:49.948585    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk" event={"ID":"dbb431a4-89ed-438b-9f24-5573b880f19c","Type":"ContainerStarted","Data":"2a630408f894d26cb431473ae6221aab71560db2f5691127cf577f38d94faf53"}
Oct 30 19:13:50.036663 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:50.036633    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:50.038355 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:50.038330    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:13:50.038475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:50.038375    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk" podUID="dbb431a4-89ed-438b-9f24-5573b880f19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:13:50.065686 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:50.065637    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk" podStartSLOduration=2.065623296 podStartE2EDuration="2.065623296s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:13:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:50.062842505 +0000 UTC m=+1603.009010421" watchObservedRunningTime="2026-10-30 19:13:50.065623296 +0000 UTC m=+1603.011791240"
Oct 30 19:13:51.040056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:51.039476    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:13:51.040056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:51.039548    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk" podUID="dbb431a4-89ed-438b-9f24-5573b880f19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:13:51.986095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:51.986057    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:52.042374 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:52.042338    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:13:52.042924 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:52.042893    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk" podUID="dbb431a4-89ed-438b-9f24-5573b880f19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:13:52.207773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:52.207733    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:53.040482 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:53.040445    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:13:53.040773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:53.040740    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk" podUID="dbb431a4-89ed-438b-9f24-5573b880f19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:13:53.087074 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:53.087044    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:53.817055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:53.795812    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:54.039947 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:54.039446    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:13:54.039947 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:54.039516    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk" podUID="dbb431a4-89ed-438b-9f24-5573b880f19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:13:54.240115 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:54.239659    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:54.429581 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:54.429437    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:55.043057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:55.042418    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:13:55.043057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:55.042495    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk" podUID="dbb431a4-89ed-438b-9f24-5573b880f19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:13:55.404771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:55.404542    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:55.608081 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:55.600682    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:56.110059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.085515    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Oct 30 19:13:56.138055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.110602    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk" podUID="dbb431a4-89ed-438b-9f24-5573b880f19c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Oct 30 19:13:56.138338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.136377    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"]
Oct 30 19:13:56.144057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.144011    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"
Oct 30 19:13:56.188607 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.188571    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"]
Oct 30 19:13:56.229119 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.229086    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-f1b38f-dockercfg-tpdd2\""
Oct 30 19:13:56.229119 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.229100    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-f1b38f\""
Oct 30 19:13:56.305201 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.305165    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/372bbc79-40bf-481b-be1b-be32a5a7f7ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw\" (UID: \"372bbc79-40bf-481b-be1b-be32a5a7f7ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"
Oct 30 19:13:56.306675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.306644    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts527\" (UniqueName: \"kubernetes.io/projected/372bbc79-40bf-481b-be1b-be32a5a7f7ca-kube-api-access-ts527\") pod \"payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw\" (UID: \"372bbc79-40bf-481b-be1b-be32a5a7f7ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"
Oct 30 19:13:56.408078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.407994    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/372bbc79-40bf-481b-be1b-be32a5a7f7ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw\" (UID: \"372bbc79-40bf-481b-be1b-be32a5a7f7ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"
Oct 30 19:13:56.408469 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.408223    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ts527\" (UniqueName: \"kubernetes.io/projected/372bbc79-40bf-481b-be1b-be32a5a7f7ca-kube-api-access-ts527\") pod \"payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw\" (UID: \"372bbc79-40bf-481b-be1b-be32a5a7f7ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"
Oct 30 19:13:56.408725 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.408707    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/372bbc79-40bf-481b-be1b-be32a5a7f7ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw\" (UID: \"372bbc79-40bf-481b-be1b-be32a5a7f7ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"
Oct 30 19:13:56.466296 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.466250    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts527\" (UniqueName: \"kubernetes.io/projected/372bbc79-40bf-481b-be1b-be32a5a7f7ca-kube-api-access-ts527\") pod \"payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw\" (UID: \"372bbc79-40bf-481b-be1b-be32a5a7f7ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"
Oct 30 19:13:56.501269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.501240    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"
Oct 30 19:13:56.885000 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:56.884826    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"]
Oct 30 19:13:56.891630 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:13:56.891594    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod372bbc79_40bf_481b_be1b_be32a5a7f7ca.slice/crio-fff59cf499868c67330d5c8569a9bb67e074e1ea73c68ee6ef6aeb802ab0609b WatchSource:0}: Error finding container fff59cf499868c67330d5c8569a9bb67e074e1ea73c68ee6ef6aeb802ab0609b: Status 404 returned error can't find the container with id fff59cf499868c67330d5c8569a9bb67e074e1ea73c68ee6ef6aeb802ab0609b
Oct 30 19:13:57.041300 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:57.041274    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:57.041727 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:57.041710    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:57.042368 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:57.042353    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:13:57.058665 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:57.058446    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw" event={"ID":"372bbc79-40bf-481b-be1b-be32a5a7f7ca","Type":"ContainerStarted","Data":"ca2bd9277d231dc187024789afdbd2b5eb43deaf3709120a9e8526bdfa1522cc"}
Oct 30 19:13:57.058665 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:57.058493    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw" event={"ID":"372bbc79-40bf-481b-be1b-be32a5a7f7ca","Type":"ContainerStarted","Data":"fff59cf499868c67330d5c8569a9bb67e074e1ea73c68ee6ef6aeb802ab0609b"}
Oct 30 19:13:57.058825 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:57.058727    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"
Oct 30 19:13:57.272953 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:57.269540    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw" podStartSLOduration=1.269519137 podStartE2EDuration="1.269519137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:13:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:57.268930688 +0000 UTC m=+1610.215098620" watchObservedRunningTime="2026-10-30 19:13:57.269519137 +0000 UTC m=+1610.215687040"
Oct 30 19:13:58.023845 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:58.023813    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:58.218202 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:58.218165    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:58.416434 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:58.416349    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:13:59.448471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:13:59.446700    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:01.214303 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:01.214269    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:01.474104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:01.471766    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:01.804054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:01.800280    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"]
Oct 30 19:14:01.838969 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:01.834552    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"
Oct 30 19:14:01.848485 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:01.848450    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7nzn\" (UniqueName: \"kubernetes.io/projected/2a762f75-0170-4a91-bcf4-bb2af0079517-kube-api-access-q7nzn\") pod \"payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp\" (UID: \"2a762f75-0170-4a91-bcf4-bb2af0079517\") " pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"
Oct 30 19:14:01.848858 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:01.848834    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a762f75-0170-4a91-bcf4-bb2af0079517-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp\" (UID: \"2a762f75-0170-4a91-bcf4-bb2af0079517\") " pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"
Oct 30 19:14:01.897067 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:01.896913    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-f44636\""
Oct 30 19:14:01.897273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:01.897211    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-f44636-dockercfg-cq9rj\""
Oct 30 19:14:01.965124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:01.963839    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a762f75-0170-4a91-bcf4-bb2af0079517-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp\" (UID: \"2a762f75-0170-4a91-bcf4-bb2af0079517\") " pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"
Oct 30 19:14:01.965124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:01.964178    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7nzn\" (UniqueName: \"kubernetes.io/projected/2a762f75-0170-4a91-bcf4-bb2af0079517-kube-api-access-q7nzn\") pod \"payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp\" (UID: \"2a762f75-0170-4a91-bcf4-bb2af0079517\") " pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"
Oct 30 19:14:01.966997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:01.966207    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a762f75-0170-4a91-bcf4-bb2af0079517-plugins-config-volume\") pod \"payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp\" (UID: \"2a762f75-0170-4a91-bcf4-bb2af0079517\") " pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"
Oct 30 19:14:01.973299 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:01.973269    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"]
Oct 30 19:14:02.014150 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.013940    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"]
Oct 30 19:14:02.020382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.020235    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"
Oct 30 19:14:02.020692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.020643    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"]
Oct 30 19:14:02.033696 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.033666    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7nzn\" (UniqueName: \"kubernetes.io/projected/2a762f75-0170-4a91-bcf4-bb2af0079517-kube-api-access-q7nzn\") pod \"payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp\" (UID: \"2a762f75-0170-4a91-bcf4-bb2af0079517\") " pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"
Oct 30 19:14:02.065864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.065802    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l\" (UID: \"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"
Oct 30 19:14:02.066014 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.065993    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klt6w\" (UniqueName: \"kubernetes.io/projected/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-kube-api-access-klt6w\") pod \"payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l\" (UID: \"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"
Oct 30 19:14:02.166883 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.166853    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l\" (UID: \"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"
Oct 30 19:14:02.167062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.167025    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klt6w\" (UniqueName: \"kubernetes.io/projected/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-kube-api-access-klt6w\") pod \"payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l\" (UID: \"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"
Oct 30 19:14:02.167541 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.167522    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l\" (UID: \"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"
Oct 30 19:14:02.226161 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.226136    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-klt6w\" (UniqueName: \"kubernetes.io/projected/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-kube-api-access-klt6w\") pod \"payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l\" (UID: \"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"
Oct 30 19:14:02.268622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.268577    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"
Oct 30 19:14:02.339700 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.339349    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"
Oct 30 19:14:02.643069 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.642986    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"]
Oct 30 19:14:02.648866 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:14:02.648833    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a762f75_0170_4a91_bcf4_bb2af0079517.slice/crio-7fa608169472c9f108baab13e78ebfd84ac8129f8da0158b9b68148bea1ec35b WatchSource:0}: Error finding container 7fa608169472c9f108baab13e78ebfd84ac8129f8da0158b9b68148bea1ec35b: Status 404 returned error can't find the container with id 7fa608169472c9f108baab13e78ebfd84ac8129f8da0158b9b68148bea1ec35b
Oct 30 19:14:02.748028 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:02.748004    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"]
Oct 30 19:14:02.754023 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:14:02.753992    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe2b1ef6_71e9_470a_ae30_aa957f4c84b6.slice/crio-0bde6152a95abb995729fcefadf5d66f58a94de925325e22655e78fb7a36e306 WatchSource:0}: Error finding container 0bde6152a95abb995729fcefadf5d66f58a94de925325e22655e78fb7a36e306: Status 404 returned error can't find the container with id 0bde6152a95abb995729fcefadf5d66f58a94de925325e22655e78fb7a36e306
Oct 30 19:14:03.096918 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:03.096874    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l" event={"ID":"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6","Type":"ContainerStarted","Data":"6034faac46dbb2e19786f3c935d9c7fcc1e5ad2585320a88ad1d03fd13223356"}
Oct 30 19:14:03.097112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:03.096935    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l" event={"ID":"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6","Type":"ContainerStarted","Data":"0bde6152a95abb995729fcefadf5d66f58a94de925325e22655e78fb7a36e306"}
Oct 30 19:14:03.097112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:03.097104    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"
Oct 30 19:14:03.098745 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:03.098713    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp" event={"ID":"2a762f75-0170-4a91-bcf4-bb2af0079517","Type":"ContainerStarted","Data":"e7a19155d6ba03a402f4ebee49daa76279b06131513998f8cf2103d3549416db"}
Oct 30 19:14:03.098866 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:03.098758    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp" event={"ID":"2a762f75-0170-4a91-bcf4-bb2af0079517","Type":"ContainerStarted","Data":"7fa608169472c9f108baab13e78ebfd84ac8129f8da0158b9b68148bea1ec35b"}
Oct 30 19:14:03.098866 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:03.098788    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"
Oct 30 19:14:03.211269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:03.211225    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l" podStartSLOduration=2.211211388 podStartE2EDuration="2.211211388s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:03.208649752 +0000 UTC m=+1616.154817656" watchObservedRunningTime="2026-10-30 19:14:03.211211388 +0000 UTC m=+1616.157379290"
Oct 30 19:14:03.268552 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:03.268521    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:03.345763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:03.341303    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp" podStartSLOduration=2.341281071 podStartE2EDuration="2.341281071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:03.335025982 +0000 UTC m=+1616.281193886" watchObservedRunningTime="2026-10-30 19:14:03.341281071 +0000 UTC m=+1616.287448976"
Oct 30 19:14:03.463960 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:14:03.463844    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:14:04.966854 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:04.966817    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:08.067941 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:08.067911    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"
Oct 30 19:14:08.120171 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:08.120140    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:08.301474 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:08.301433    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:08.903676 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:08.903647    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:14:09.727487 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:09.727448    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:10.739655 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:10.739608    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:11.698626 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:11.698590    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:14.107847 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:14.107810    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"
Oct 30 19:14:14.108328 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:14.107914    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"
Oct 30 19:14:14.635473 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:14.635427    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:19.203234 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:19.198091    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:23.998157 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:23.991140    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:25.937066 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:25.937010    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:26.703648 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:26.703613    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:27.445066 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:27.433123    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:27.761048 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:27.760984    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:29.194735 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:29.194684    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:31.516611 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:31.516551    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:31.704705 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:31.704670    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:33.385126 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:14:33.385089    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:14:34.094225 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:34.094181    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:34.602820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:34.602787    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:34.914637 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:34.914548    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:35.015988 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:35.014577    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"]
Oct 30 19:14:35.027244 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:35.027210    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"
Oct 30 19:14:35.081092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:35.078952    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-0d2ef6-dockercfg-v425b\""
Oct 30 19:14:35.084640 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:35.084609    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"]
Oct 30 19:14:35.085064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:35.085043    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-0d2ef6\""
Oct 30 19:14:35.226110 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:35.216544    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzdd6\" (UniqueName: \"kubernetes.io/projected/be13b502-20cc-4754-ab38-71eee24dca18-kube-api-access-wzdd6\") pod \"payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl\" (UID: \"be13b502-20cc-4754-ab38-71eee24dca18\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"
Oct 30 19:14:35.226110 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:35.216952    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be13b502-20cc-4754-ab38-71eee24dca18-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl\" (UID: \"be13b502-20cc-4754-ab38-71eee24dca18\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"
Oct 30 19:14:35.355131 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:35.348818    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be13b502-20cc-4754-ab38-71eee24dca18-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl\" (UID: \"be13b502-20cc-4754-ab38-71eee24dca18\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"
Oct 30 19:14:35.355131 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:35.349109    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzdd6\" (UniqueName: \"kubernetes.io/projected/be13b502-20cc-4754-ab38-71eee24dca18-kube-api-access-wzdd6\") pod \"payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl\" (UID: \"be13b502-20cc-4754-ab38-71eee24dca18\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"
Oct 30 19:14:35.355131 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:35.350899    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be13b502-20cc-4754-ab38-71eee24dca18-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl\" (UID: \"be13b502-20cc-4754-ab38-71eee24dca18\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"
Oct 30 19:14:35.431091 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:35.425815    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzdd6\" (UniqueName: \"kubernetes.io/projected/be13b502-20cc-4754-ab38-71eee24dca18-kube-api-access-wzdd6\") pod \"payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl\" (UID: \"be13b502-20cc-4754-ab38-71eee24dca18\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"
Oct 30 19:14:35.711411 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:35.711374    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"
Oct 30 19:14:36.090527 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:36.090498    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"]
Oct 30 19:14:36.100098 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:14:36.100072    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe13b502_20cc_4754_ab38_71eee24dca18.slice/crio-86b4af309d61ed0c2b9e8aad8d0f98a499e02b1a191e5b2bedb90bb6e01e11a9 WatchSource:0}: Error finding container 86b4af309d61ed0c2b9e8aad8d0f98a499e02b1a191e5b2bedb90bb6e01e11a9: Status 404 returned error can't find the container with id 86b4af309d61ed0c2b9e8aad8d0f98a499e02b1a191e5b2bedb90bb6e01e11a9
Oct 30 19:14:36.439178 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:36.439102    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl" event={"ID":"be13b502-20cc-4754-ab38-71eee24dca18","Type":"ContainerStarted","Data":"66b2d023282ca244a3ea84be2c30f3571516332b33e0b3efb7e023593d21de73"}
Oct 30 19:14:36.439178 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:36.439155    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl" event={"ID":"be13b502-20cc-4754-ab38-71eee24dca18","Type":"ContainerStarted","Data":"86b4af309d61ed0c2b9e8aad8d0f98a499e02b1a191e5b2bedb90bb6e01e11a9"}
Oct 30 19:14:36.439369 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:36.439236    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"
Oct 30 19:14:36.550042 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:36.549983    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl" podStartSLOduration=2.549969862 podStartE2EDuration="2.549969862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:36.547937082 +0000 UTC m=+1649.494104987" watchObservedRunningTime="2026-10-30 19:14:36.549969862 +0000 UTC m=+1649.496137813"
Oct 30 19:14:38.768816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:38.768777    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"]
Oct 30 19:14:38.773907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:38.773888    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:38.822702 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:38.822683    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-f1b38f-serving-cert\""
Oct 30 19:14:38.872005 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:38.871980    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"]
Oct 30 19:14:38.911775 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:38.911739    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-openshift-service-ca\") pod \"maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:38.911954 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:38.911922    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-metrics-certs\") pod \"maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:38.912085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:38.912017    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-maas-api-tls\") pod \"maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:38.912355 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:38.912333    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64c86\" (UniqueName: \"kubernetes.io/projected/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-kube-api-access-64c86\") pod \"maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:39.005646 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:39.005610    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:39.014135 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:39.014104    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-openshift-service-ca\") pod \"maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:39.014328 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:39.014218    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-metrics-certs\") pod \"maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:39.014328 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:39.014313    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-maas-api-tls\") pod \"maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:39.014490 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:39.014397    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64c86\" (UniqueName: \"kubernetes.io/projected/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-kube-api-access-64c86\") pod \"maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:39.014883 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:14:39.014864    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-serving-cert: secret "maas-api-e2e-api-b-f1b38f-serving-cert" not found
Oct 30 19:14:39.015117 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:14:39.015102    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-maas-api-tls podName:1324e64f-d43b-46dc-bcbf-7d5d4565b3df nodeName:}" failed. No retries permitted until 2026-10-30 19:14:39.515078108 +0000 UTC m=+1652.461246005 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-maas-api-tls") pod "maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v" (UID: "1324e64f-d43b-46dc-bcbf-7d5d4565b3df") : secret "maas-api-e2e-api-b-f1b38f-serving-cert" not found
Oct 30 19:14:39.019148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:39.019088    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-metrics-certs\") pod \"maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:39.044140 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:39.033299    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-openshift-service-ca\") pod \"maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:39.075930 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:39.075897    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-64c86\" (UniqueName: \"kubernetes.io/projected/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-kube-api-access-64c86\") pod \"maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:39.236976 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:39.236932    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:39.475214 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:39.475134    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:39.525735 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:39.525703    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-maas-api-tls\") pod \"maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:39.528093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:39.528029    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-maas-api-tls\") pod \"maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:39.688815 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:39.688784    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:40.069437 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:40.069401    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"]
Oct 30 19:14:40.074481 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:14:40.074454    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1324e64f_d43b_46dc_bcbf_7d5d4565b3df.slice/crio-671aabdd69a51296eaf07df0c41971af34d6ff113507d1ddf3608f331b62d1f9 WatchSource:0}: Error finding container 671aabdd69a51296eaf07df0c41971af34d6ff113507d1ddf3608f331b62d1f9: Status 404 returned error can't find the container with id 671aabdd69a51296eaf07df0c41971af34d6ff113507d1ddf3608f331b62d1f9
Oct 30 19:14:40.468676 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:40.468585    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v" event={"ID":"1324e64f-d43b-46dc-bcbf-7d5d4565b3df","Type":"ContainerStarted","Data":"671aabdd69a51296eaf07df0c41971af34d6ff113507d1ddf3608f331b62d1f9"}
Oct 30 19:14:40.672734 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:40.672701    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:14:41.479209 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:41.479168    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v" event={"ID":"1324e64f-d43b-46dc-bcbf-7d5d4565b3df","Type":"ContainerStarted","Data":"390629f3a8f08e14964954b4b1073279e6c03bb431d5f21f1279bbc8d289d956"}
Oct 30 19:14:41.479693 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:41.479323    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:41.591251 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:41.591197    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v" podStartSLOduration=2.690307877 podStartE2EDuration="3.591184266s" totalImagesPullingTime="900.876389ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:41.590959831 +0000 UTC m=+1654.537127748" watchObservedRunningTime="2026-10-30 19:14:41.591184266 +0000 UTC m=+1654.537352170"
Oct 30 19:14:44.053644 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:44.053365    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:47.448602 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:47.448569    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"
Oct 30 19:14:47.501806 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:47.501781    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:14:48.626222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:48.626191    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:48.876421 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:48.876256    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:53.496059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:53.489434    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:58.152212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:58.152167    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:58.516965 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:58.516934    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:14:58.711475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:14:58.711438    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:02.911627 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:02.911587    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:03.383891 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:15:03.383851    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:15:03.660628 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:03.660546    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:07.608058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:07.601352    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:09.428392 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:09.428354    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:12.186579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:12.186543    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:16.846105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:16.845763    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:19.773878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:19.773841    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:21.393061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:21.392582    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:25.686325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.686293    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"]
Oct 30 19:15:25.692222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.692200    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:25.747862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.747836    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-0d2ef6-serving-cert\""
Oct 30 19:15:25.765073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.765027    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9pnk\" (UniqueName: \"kubernetes.io/projected/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-kube-api-access-k9pnk\") pod \"maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:25.765199 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.765164    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-openshift-service-ca\") pod \"maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:25.765276 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.765260    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-maas-api-tls\") pod \"maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:25.765367 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.765347    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-metrics-certs\") pod \"maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:25.799018 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.798989    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"]
Oct 30 19:15:25.867076 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.867044    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-maas-api-tls\") pod \"maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:25.867248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.867195    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-metrics-certs\") pod \"maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:25.867248 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:15:25.867231    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-serving-cert: secret "maas-api-e2e-api-b-0d2ef6-serving-cert" not found
Oct 30 19:15:25.867371 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:15:25.867310    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-maas-api-tls podName:c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884 nodeName:}" failed. No retries permitted until 2026-10-30 19:15:26.36728935 +0000 UTC m=+1699.313457233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-maas-api-tls") pod "maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k" (UID: "c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884") : secret "maas-api-e2e-api-b-0d2ef6-serving-cert" not found
Oct 30 19:15:25.867371 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.867361    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9pnk\" (UniqueName: \"kubernetes.io/projected/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-kube-api-access-k9pnk\") pod \"maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:25.867489 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.867451    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-openshift-service-ca\") pod \"maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:25.868343 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.868316    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-openshift-service-ca\") pod \"maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:25.870551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.870526    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-metrics-certs\") pod \"maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:25.932466 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.932440    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9pnk\" (UniqueName: \"kubernetes.io/projected/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-kube-api-access-k9pnk\") pod \"maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:25.968650 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:25.968588    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:26.217361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:26.217317    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:26.382557 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:26.382526    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-maas-api-tls\") pod \"maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:26.385201 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:26.385170    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-maas-api-tls\") pod \"maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:26.606146 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:26.606111    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:26.987789 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:26.987700    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"]
Oct 30 19:15:26.994660 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:15:26.994632    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc10a8ae1_4bbe_4b2e_817a_01ebfd1fa884.slice/crio-91ae6a7d3fb5903289fcc0963fffebb402a421663605b5a3909d5e9eb1aaae3d WatchSource:0}: Error finding container 91ae6a7d3fb5903289fcc0963fffebb402a421663605b5a3909d5e9eb1aaae3d: Status 404 returned error can't find the container with id 91ae6a7d3fb5903289fcc0963fffebb402a421663605b5a3909d5e9eb1aaae3d
Oct 30 19:15:28.006763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:28.006732    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k" event={"ID":"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884","Type":"ContainerStarted","Data":"eb788f66ccbb4b911f4ac4d1f9ea60821997a6c131bdb598d3aeaa3f8506cbbc"}
Oct 30 19:15:28.007190 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:28.006810    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k" event={"ID":"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884","Type":"ContainerStarted","Data":"91ae6a7d3fb5903289fcc0963fffebb402a421663605b5a3909d5e9eb1aaae3d"}
Oct 30 19:15:28.007190 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:28.007004    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:28.126271 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:28.126186    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k" podStartSLOduration=2.290588385 podStartE2EDuration="3.126173291s" totalImagesPullingTime="835.584906ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:28.125099952 +0000 UTC m=+1701.071267856" watchObservedRunningTime="2026-10-30 19:15:28.126173291 +0000 UTC m=+1701.072341196"
Oct 30 19:15:29.200300 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:29.200269    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:15:31.293059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:31.290065    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:33.384207 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:15:33.384171    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:15:34.030148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:34.030121    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:15:36.186275 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:36.186239    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:37.793985 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:37.793955    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"]
Oct 30 19:15:38.241841 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:38.241754    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:38.443635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:38.440194    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:41.082444 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:41.082407    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:41.346899 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:41.346810    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:42.795232 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:42.795198    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5" podUID="e8c94af9-b84b-45e0-9915-e42882f53cf6" containerName="payload-processing" containerID="cri-o://7ffa21adc09de2c7a7a440e1043418b2f4b040c27d58a1a161a6de910adda7de" gracePeriod=25
Oct 30 19:15:44.076212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:44.076170    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:44.340665 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:44.340576    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5" podUID="e8c94af9-b84b-45e0-9915-e42882f53cf6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:46.264407 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:46.264364    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:48.197773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:48.197677    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:51.364535 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:51.364494    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:54.340491 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:54.340458    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5" podUID="e8c94af9-b84b-45e0-9915-e42882f53cf6" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:15:55.230779 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:55.230745    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:56.058918 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:56.058874    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:56.331176 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:56.330973    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:56.711876 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:56.711783    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:57.245323 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:57.245287    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:57.355789 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:57.355754    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"]
Oct 30 19:15:57.457096 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:57.457055    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"]
Oct 30 19:15:57.739880 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:57.739848    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:15:57.925070 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:15:57.917845    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:01.317075 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:01.315501    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:02.356715 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.356661    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q" podUID="20503cda-6f8a-4b59-8027-462e65a4a5ef" containerName="payload-processing" containerID="cri-o://916588c3a7a03eba98ff1c9a8086200923ca4b37c99f21427488a2119aeceb99" gracePeriod=25
Oct 30 19:16:02.458095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.458026    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz" podUID="d696b299-3a8b-49a0-be2f-127c6967ecc0" containerName="payload-pre-processing" containerID="cri-o://540407c373260223f9dee312723d3b903e0adb5431b202b9f381e6fe0b16c8e6" gracePeriod=25
Oct 30 19:16:02.642525 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.642500    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"
Oct 30 19:16:02.686060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.685958    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20503cda-6f8a-4b59-8027-462e65a4a5ef-kube-api-access-tqf8q\" (UniqueName: \"kubernetes.io/projected/20503cda-6f8a-4b59-8027-462e65a4a5ef-kube-api-access-tqf8q\") pod \"20503cda-6f8a-4b59-8027-462e65a4a5ef\" (UID: \"20503cda-6f8a-4b59-8027-462e65a4a5ef\") "
Oct 30 19:16:02.686060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.685998    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20503cda-6f8a-4b59-8027-462e65a4a5ef-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20503cda-6f8a-4b59-8027-462e65a4a5ef-plugins-config-volume\") pod \"20503cda-6f8a-4b59-8027-462e65a4a5ef\" (UID: \"20503cda-6f8a-4b59-8027-462e65a4a5ef\") "
Oct 30 19:16:02.687163 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.686532    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20503cda-6f8a-4b59-8027-462e65a4a5ef-plugins-config-volume" pod "20503cda-6f8a-4b59-8027-462e65a4a5ef" (UID: "20503cda-6f8a-4b59-8027-462e65a4a5ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:02.687163 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.687112    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20503cda-6f8a-4b59-8027-462e65a4a5ef-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:02.688796 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.688763    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20503cda-6f8a-4b59-8027-462e65a4a5ef-kube-api-access-tqf8q" pod "20503cda-6f8a-4b59-8027-462e65a4a5ef" (UID: "20503cda-6f8a-4b59-8027-462e65a4a5ef"). InnerVolumeSpecName "kube-api-access-tqf8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:02.752011 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.751988    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"
Oct 30 19:16:02.788606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.788576    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d696b299-3a8b-49a0-be2f-127c6967ecc0-kube-api-access-dbqrq\" (UniqueName: \"kubernetes.io/projected/d696b299-3a8b-49a0-be2f-127c6967ecc0-kube-api-access-dbqrq\") pod \"d696b299-3a8b-49a0-be2f-127c6967ecc0\" (UID: \"d696b299-3a8b-49a0-be2f-127c6967ecc0\") "
Oct 30 19:16:02.788925 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.788907    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d696b299-3a8b-49a0-be2f-127c6967ecc0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d696b299-3a8b-49a0-be2f-127c6967ecc0-plugins-config-volume\") pod \"d696b299-3a8b-49a0-be2f-127c6967ecc0\" (UID: \"d696b299-3a8b-49a0-be2f-127c6967ecc0\") "
Oct 30 19:16:02.790148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.789347    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d696b299-3a8b-49a0-be2f-127c6967ecc0-plugins-config-volume" pod "d696b299-3a8b-49a0-be2f-127c6967ecc0" (UID: "d696b299-3a8b-49a0-be2f-127c6967ecc0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:02.790148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.789528    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d696b299-3a8b-49a0-be2f-127c6967ecc0-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:02.790148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.789549    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqf8q\" (UniqueName: \"kubernetes.io/projected/20503cda-6f8a-4b59-8027-462e65a4a5ef-kube-api-access-tqf8q\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:02.791371 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.791350    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d696b299-3a8b-49a0-be2f-127c6967ecc0-kube-api-access-dbqrq" pod "d696b299-3a8b-49a0-be2f-127c6967ecc0" (UID: "d696b299-3a8b-49a0-be2f-127c6967ecc0"). InnerVolumeSpecName "kube-api-access-dbqrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:02.890903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:02.890837    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dbqrq\" (UniqueName: \"kubernetes.io/projected/d696b299-3a8b-49a0-be2f-127c6967ecc0-kube-api-access-dbqrq\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:03.043633 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.043606    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"
Oct 30 19:16:03.094494 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.094466    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e8c94af9-b84b-45e0-9915-e42882f53cf6-kube-api-access-zcxzr\" (UniqueName: \"kubernetes.io/projected/e8c94af9-b84b-45e0-9915-e42882f53cf6-kube-api-access-zcxzr\") pod \"e8c94af9-b84b-45e0-9915-e42882f53cf6\" (UID: \"e8c94af9-b84b-45e0-9915-e42882f53cf6\") "
Oct 30 19:16:03.094652 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.094639    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e8c94af9-b84b-45e0-9915-e42882f53cf6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c94af9-b84b-45e0-9915-e42882f53cf6-plugins-config-volume\") pod \"e8c94af9-b84b-45e0-9915-e42882f53cf6\" (UID: \"e8c94af9-b84b-45e0-9915-e42882f53cf6\") "
Oct 30 19:16:03.094969 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.094947    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8c94af9-b84b-45e0-9915-e42882f53cf6-plugins-config-volume" pod "e8c94af9-b84b-45e0-9915-e42882f53cf6" (UID: "e8c94af9-b84b-45e0-9915-e42882f53cf6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:03.095187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.095170    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8c94af9-b84b-45e0-9915-e42882f53cf6-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:03.096521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.096497    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c94af9-b84b-45e0-9915-e42882f53cf6-kube-api-access-zcxzr" pod "e8c94af9-b84b-45e0-9915-e42882f53cf6" (UID: "e8c94af9-b84b-45e0-9915-e42882f53cf6"). InnerVolumeSpecName "kube-api-access-zcxzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:03.196297 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.196271    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcxzr\" (UniqueName: \"kubernetes.io/projected/e8c94af9-b84b-45e0-9915-e42882f53cf6-kube-api-access-zcxzr\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:03.355697 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.355663    5704 generic.go:356] "Generic (PLEG): container finished" podID="d696b299-3a8b-49a0-be2f-127c6967ecc0" containerID="540407c373260223f9dee312723d3b903e0adb5431b202b9f381e6fe0b16c8e6" exitCode=0
Oct 30 19:16:03.355866 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.355708    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"
Oct 30 19:16:03.355866 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.355740    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz" event={"ID":"d696b299-3a8b-49a0-be2f-127c6967ecc0","Type":"ContainerDied","Data":"540407c373260223f9dee312723d3b903e0adb5431b202b9f381e6fe0b16c8e6"}
Oct 30 19:16:03.355866 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.355794    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz" event={"ID":"d696b299-3a8b-49a0-be2f-127c6967ecc0","Type":"ContainerDied","Data":"bb11239d1b5ee52d4c6833801e0627162adc1d1b2bdd415c32f8934fe5288cb2"}
Oct 30 19:16:03.355866 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.355823    5704 scope.go:122] "RemoveContainer" containerID="540407c373260223f9dee312723d3b903e0adb5431b202b9f381e6fe0b16c8e6"
Oct 30 19:16:03.358556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.358431    5704 generic.go:356] "Generic (PLEG): container finished" podID="20503cda-6f8a-4b59-8027-462e65a4a5ef" containerID="916588c3a7a03eba98ff1c9a8086200923ca4b37c99f21427488a2119aeceb99" exitCode=0
Oct 30 19:16:03.358556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.358499    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"
Oct 30 19:16:03.358932 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.358557    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q" event={"ID":"20503cda-6f8a-4b59-8027-462e65a4a5ef","Type":"ContainerDied","Data":"916588c3a7a03eba98ff1c9a8086200923ca4b37c99f21427488a2119aeceb99"}
Oct 30 19:16:03.358932 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.358608    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q" event={"ID":"20503cda-6f8a-4b59-8027-462e65a4a5ef","Type":"ContainerDied","Data":"be58e10e31b65ff6f66dc10bdede9a67c1ebcd6b8f2acf11202fd7788567f8f3"}
Oct 30 19:16:03.360832 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.360808    5704 generic.go:356] "Generic (PLEG): container finished" podID="e8c94af9-b84b-45e0-9915-e42882f53cf6" containerID="7ffa21adc09de2c7a7a440e1043418b2f4b040c27d58a1a161a6de910adda7de" exitCode=0
Oct 30 19:16:03.360979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.360844    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5" event={"ID":"e8c94af9-b84b-45e0-9915-e42882f53cf6","Type":"ContainerDied","Data":"7ffa21adc09de2c7a7a440e1043418b2f4b040c27d58a1a161a6de910adda7de"}
Oct 30 19:16:03.360979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.360867    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"
Oct 30 19:16:03.360979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.360889    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5" event={"ID":"e8c94af9-b84b-45e0-9915-e42882f53cf6","Type":"ContainerDied","Data":"9e7c1e599dd30fd0c67b83d6e542f1053b2a1e657d0991e7145d7975778318be"}
Oct 30 19:16:03.388653 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:16:03.388626    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:16:03.399539 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.399519    5704 scope.go:122] "RemoveContainer" containerID="540407c373260223f9dee312723d3b903e0adb5431b202b9f381e6fe0b16c8e6"
Oct 30 19:16:03.399869 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:16:03.399848    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"540407c373260223f9dee312723d3b903e0adb5431b202b9f381e6fe0b16c8e6\": container with ID starting with 540407c373260223f9dee312723d3b903e0adb5431b202b9f381e6fe0b16c8e6 not found: ID does not exist" containerID="540407c373260223f9dee312723d3b903e0adb5431b202b9f381e6fe0b16c8e6"
Oct 30 19:16:03.399961 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.399876    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"540407c373260223f9dee312723d3b903e0adb5431b202b9f381e6fe0b16c8e6"} err="failed to get container status \"540407c373260223f9dee312723d3b903e0adb5431b202b9f381e6fe0b16c8e6\": rpc error: code = NotFound desc = could not find container \"540407c373260223f9dee312723d3b903e0adb5431b202b9f381e6fe0b16c8e6\": container with ID starting with 540407c373260223f9dee312723d3b903e0adb5431b202b9f381e6fe0b16c8e6 not found: ID does not exist"
Oct 30 19:16:03.399961 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.399893    5704 scope.go:122] "RemoveContainer" containerID="916588c3a7a03eba98ff1c9a8086200923ca4b37c99f21427488a2119aeceb99"
Oct 30 19:16:03.409101 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.409079    5704 scope.go:122] "RemoveContainer" containerID="916588c3a7a03eba98ff1c9a8086200923ca4b37c99f21427488a2119aeceb99"
Oct 30 19:16:03.409357 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:16:03.409337    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"916588c3a7a03eba98ff1c9a8086200923ca4b37c99f21427488a2119aeceb99\": container with ID starting with 916588c3a7a03eba98ff1c9a8086200923ca4b37c99f21427488a2119aeceb99 not found: ID does not exist" containerID="916588c3a7a03eba98ff1c9a8086200923ca4b37c99f21427488a2119aeceb99"
Oct 30 19:16:03.409432 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.409369    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"916588c3a7a03eba98ff1c9a8086200923ca4b37c99f21427488a2119aeceb99"} err="failed to get container status \"916588c3a7a03eba98ff1c9a8086200923ca4b37c99f21427488a2119aeceb99\": rpc error: code = NotFound desc = could not find container \"916588c3a7a03eba98ff1c9a8086200923ca4b37c99f21427488a2119aeceb99\": container with ID starting with 916588c3a7a03eba98ff1c9a8086200923ca4b37c99f21427488a2119aeceb99 not found: ID does not exist"
Oct 30 19:16:03.409432 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.409391    5704 scope.go:122] "RemoveContainer" containerID="7ffa21adc09de2c7a7a440e1043418b2f4b040c27d58a1a161a6de910adda7de"
Oct 30 19:16:03.417820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.417802    5704 scope.go:122] "RemoveContainer" containerID="7ffa21adc09de2c7a7a440e1043418b2f4b040c27d58a1a161a6de910adda7de"
Oct 30 19:16:03.418111 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:16:03.418093    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ffa21adc09de2c7a7a440e1043418b2f4b040c27d58a1a161a6de910adda7de\": container with ID starting with 7ffa21adc09de2c7a7a440e1043418b2f4b040c27d58a1a161a6de910adda7de not found: ID does not exist" containerID="7ffa21adc09de2c7a7a440e1043418b2f4b040c27d58a1a161a6de910adda7de"
Oct 30 19:16:03.418169 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.418116    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ffa21adc09de2c7a7a440e1043418b2f4b040c27d58a1a161a6de910adda7de"} err="failed to get container status \"7ffa21adc09de2c7a7a440e1043418b2f4b040c27d58a1a161a6de910adda7de\": rpc error: code = NotFound desc = could not find container \"7ffa21adc09de2c7a7a440e1043418b2f4b040c27d58a1a161a6de910adda7de\": container with ID starting with 7ffa21adc09de2c7a7a440e1043418b2f4b040c27d58a1a161a6de910adda7de not found: ID does not exist"
Oct 30 19:16:03.510989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.510959    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"]
Oct 30 19:16:03.515097 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.515076    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-4f4614-6f76b5fffb-4b28q"]
Oct 30 19:16:03.620116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.620091    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"]
Oct 30 19:16:03.625952 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.625929    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-dk5z5"]
Oct 30 19:16:03.730806 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.730777    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"]
Oct 30 19:16:03.736184 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:03.736157    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4f4614-848d4d54bc-l9rgz"]
Oct 30 19:16:04.053587 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:04.053550    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:05.180832 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:05.180801    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20503cda-6f8a-4b59-8027-462e65a4a5ef" path="/var/lib/kubelet/pods/20503cda-6f8a-4b59-8027-462e65a4a5ef/volumes"
Oct 30 19:16:05.181269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:05.181253    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d696b299-3a8b-49a0-be2f-127c6967ecc0" path="/var/lib/kubelet/pods/d696b299-3a8b-49a0-be2f-127c6967ecc0/volumes"
Oct 30 19:16:05.181623 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:05.181611    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e8c94af9-b84b-45e0-9915-e42882f53cf6" path="/var/lib/kubelet/pods/e8c94af9-b84b-45e0-9915-e42882f53cf6/volumes"
Oct 30 19:16:05.963311 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:05.963269    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:10.762556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:10.762520    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:11.191629 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:11.191545    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:16:11.276538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:11.276509    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:11.472345 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:11.472265    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:11.684636 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:11.684596    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:14.462140 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:14.462109    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:14.670054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:14.669996    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:15.241058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.240317    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"]
Oct 30 19:16:15.245631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.241321    5704 cpu_manager.go:414] "containerMap: removing container" podUID="d696b299-3a8b-49a0-be2f-127c6967ecc0" containerName="payload-pre-processing"
Oct 30 19:16:15.245631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.241342    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d696b299-3a8b-49a0-be2f-127c6967ecc0" containerName="payload-pre-processing"
Oct 30 19:16:15.245631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.241369    5704 cpu_manager.go:414] "containerMap: removing container" podUID="20503cda-6f8a-4b59-8027-462e65a4a5ef" containerName="payload-processing"
Oct 30 19:16:15.245631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.241377    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20503cda-6f8a-4b59-8027-462e65a4a5ef" containerName="payload-processing"
Oct 30 19:16:15.245631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.241418    5704 cpu_manager.go:414] "containerMap: removing container" podUID="e8c94af9-b84b-45e0-9915-e42882f53cf6" containerName="payload-processing"
Oct 30 19:16:15.245631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.241425    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e8c94af9-b84b-45e0-9915-e42882f53cf6" containerName="payload-processing"
Oct 30 19:16:15.245631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.244271    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="e8c94af9-b84b-45e0-9915-e42882f53cf6" containerName="payload-processing"
Oct 30 19:16:15.245631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.244309    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="d696b299-3a8b-49a0-be2f-127c6967ecc0" containerName="payload-pre-processing"
Oct 30 19:16:15.245631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.244326    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="20503cda-6f8a-4b59-8027-462e65a4a5ef" containerName="payload-processing"
Oct 30 19:16:15.250967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.250942    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"]
Oct 30 19:16:15.251287 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.251204    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"
Oct 30 19:16:15.301461 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.301440    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-2ec7ea-dockercfg-bjlw7\""
Oct 30 19:16:15.347354 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.347326    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmqlm\" (UniqueName: \"kubernetes.io/projected/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-kube-api-access-dmqlm\") pod \"payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7\" (UID: \"ecd45fb1-3b71-4953-936f-1aa47f8eebd5\") " pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"
Oct 30 19:16:15.347535 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.347516    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-2ec7ea\""
Oct 30 19:16:15.347618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.347533    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7\" (UID: \"ecd45fb1-3b71-4953-936f-1aa47f8eebd5\") " pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"
Oct 30 19:16:15.448977 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.448951    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7\" (UID: \"ecd45fb1-3b71-4953-936f-1aa47f8eebd5\") " pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"
Oct 30 19:16:15.449127 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.449113    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmqlm\" (UniqueName: \"kubernetes.io/projected/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-kube-api-access-dmqlm\") pod \"payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7\" (UID: \"ecd45fb1-3b71-4953-936f-1aa47f8eebd5\") " pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"
Oct 30 19:16:15.449732 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.449712    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7\" (UID: \"ecd45fb1-3b71-4953-936f-1aa47f8eebd5\") " pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"
Oct 30 19:16:15.506853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.506821    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmqlm\" (UniqueName: \"kubernetes.io/projected/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-kube-api-access-dmqlm\") pod \"payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7\" (UID: \"ecd45fb1-3b71-4953-936f-1aa47f8eebd5\") " pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"
Oct 30 19:16:15.571442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.571418    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"
Oct 30 19:16:15.935602 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.935508    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:15.978326 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:15.978243    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"]
Oct 30 19:16:16.478845 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:16.477500    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7" event={"ID":"ecd45fb1-3b71-4953-936f-1aa47f8eebd5","Type":"ContainerStarted","Data":"2dd8ab300a0fa17adbe8aee3836ea584aeb87ac56980f389f4665ff2eac45492"}
Oct 30 19:16:16.478845 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:16.477572    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7" event={"ID":"ecd45fb1-3b71-4953-936f-1aa47f8eebd5","Type":"ContainerStarted","Data":"ff29fb77ab4bbc891d3685a289093545e5542fa1efa53f4386e5c9f03cfead60"}
Oct 30 19:16:16.478845 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:16.478765    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"
Oct 30 19:16:16.588995 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:16.588937    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7" podStartSLOduration=2.588921155 podStartE2EDuration="2.588921155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:16:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:16:16.587192982 +0000 UTC m=+1749.533360886" watchObservedRunningTime="2026-10-30 19:16:16.588921155 +0000 UTC m=+1749.535089060"
Oct 30 19:16:20.745061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:20.732892    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:23.557239 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:23.557204    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:25.737455 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:25.734777    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:28.493187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:28.493157    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"
Oct 30 19:16:30.805349 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:30.805313    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:33.384273 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:16:33.384236    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:16:35.461018 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:35.460982    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:35.657453 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:35.657418    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:35.913597 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:35.913562    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:36.960404 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:36.960364    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"]
Oct 30 19:16:37.223601 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:37.223518    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:37.465619 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:37.465590    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:40.577326 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:40.577289    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:41.961444 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:41.961402    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn" podUID="c6a1e289-e8f5-456f-8d7c-1eca2944324b" containerName="payload-pre-processing" containerID="cri-o://ff700314c82fefa48403cf88cfb58c767d0602f7d688b5f1f3e97a2eb7adf740" gracePeriod=25
Oct 30 19:16:42.234597 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.234575    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"
Oct 30 19:16:42.305885 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.305860    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c6a1e289-e8f5-456f-8d7c-1eca2944324b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6a1e289-e8f5-456f-8d7c-1eca2944324b-plugins-config-volume\") pod \"c6a1e289-e8f5-456f-8d7c-1eca2944324b\" (UID: \"c6a1e289-e8f5-456f-8d7c-1eca2944324b\") "
Oct 30 19:16:42.306118 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.306101    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c6a1e289-e8f5-456f-8d7c-1eca2944324b-kube-api-access-lqgfp\" (UniqueName: \"kubernetes.io/projected/c6a1e289-e8f5-456f-8d7c-1eca2944324b-kube-api-access-lqgfp\") pod \"c6a1e289-e8f5-456f-8d7c-1eca2944324b\" (UID: \"c6a1e289-e8f5-456f-8d7c-1eca2944324b\") "
Oct 30 19:16:42.306274 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.306251    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c6a1e289-e8f5-456f-8d7c-1eca2944324b-plugins-config-volume" pod "c6a1e289-e8f5-456f-8d7c-1eca2944324b" (UID: "c6a1e289-e8f5-456f-8d7c-1eca2944324b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:42.306580 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.306566    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c6a1e289-e8f5-456f-8d7c-1eca2944324b-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:42.308172 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.308151    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6a1e289-e8f5-456f-8d7c-1eca2944324b-kube-api-access-lqgfp" pod "c6a1e289-e8f5-456f-8d7c-1eca2944324b" (UID: "c6a1e289-e8f5-456f-8d7c-1eca2944324b"). InnerVolumeSpecName "kube-api-access-lqgfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:42.407437 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.407401    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqgfp\" (UniqueName: \"kubernetes.io/projected/c6a1e289-e8f5-456f-8d7c-1eca2944324b-kube-api-access-lqgfp\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:42.694102 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.694068    5704 generic.go:356] "Generic (PLEG): container finished" podID="c6a1e289-e8f5-456f-8d7c-1eca2944324b" containerID="ff700314c82fefa48403cf88cfb58c767d0602f7d688b5f1f3e97a2eb7adf740" exitCode=0
Oct 30 19:16:42.694308 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.694118    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"
Oct 30 19:16:42.694308 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.694158    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn" event={"ID":"c6a1e289-e8f5-456f-8d7c-1eca2944324b","Type":"ContainerDied","Data":"ff700314c82fefa48403cf88cfb58c767d0602f7d688b5f1f3e97a2eb7adf740"}
Oct 30 19:16:42.694308 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.694213    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn" event={"ID":"c6a1e289-e8f5-456f-8d7c-1eca2944324b","Type":"ContainerDied","Data":"43c5e69e6a0e1ee36e7f85c41ca992211d61f077704a559f101c65bc576aebce"}
Oct 30 19:16:42.694308 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.694242    5704 scope.go:122] "RemoveContainer" containerID="ff700314c82fefa48403cf88cfb58c767d0602f7d688b5f1f3e97a2eb7adf740"
Oct 30 19:16:42.704462 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.704446    5704 scope.go:122] "RemoveContainer" containerID="ff700314c82fefa48403cf88cfb58c767d0602f7d688b5f1f3e97a2eb7adf740"
Oct 30 19:16:42.704742 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:16:42.704726    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff700314c82fefa48403cf88cfb58c767d0602f7d688b5f1f3e97a2eb7adf740\": container with ID starting with ff700314c82fefa48403cf88cfb58c767d0602f7d688b5f1f3e97a2eb7adf740 not found: ID does not exist" containerID="ff700314c82fefa48403cf88cfb58c767d0602f7d688b5f1f3e97a2eb7adf740"
Oct 30 19:16:42.704791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.704750    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff700314c82fefa48403cf88cfb58c767d0602f7d688b5f1f3e97a2eb7adf740"} err="failed to get container status \"ff700314c82fefa48403cf88cfb58c767d0602f7d688b5f1f3e97a2eb7adf740\": rpc error: code = NotFound desc = could not find container \"ff700314c82fefa48403cf88cfb58c767d0602f7d688b5f1f3e97a2eb7adf740\": container with ID starting with ff700314c82fefa48403cf88cfb58c767d0602f7d688b5f1f3e97a2eb7adf740 not found: ID does not exist"
Oct 30 19:16:42.850541 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.850512    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"]
Oct 30 19:16:42.895399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:42.895375    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-c4ca9a-7c67b99887-542dn"]
Oct 30 19:16:43.180565 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:43.180536    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c6a1e289-e8f5-456f-8d7c-1eca2944324b" path="/var/lib/kubelet/pods/c6a1e289-e8f5-456f-8d7c-1eca2944324b/volumes"
Oct 30 19:16:45.301922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:45.300117    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:48.858574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:48.858534    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"]
Oct 30 19:16:48.859066 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:48.858783    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerName="istio-proxy" containerID="cri-o://e66308ee491db3d7ac854961c995ea804d1c6d94561d4d13e25652b84ba2b109" gracePeriod=30
Oct 30 19:16:48.904444 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:48.904411    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"]
Oct 30 19:16:48.904614 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:48.904594    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r" podUID="1b169d63-af95-42fe-bcca-457d1047fcbf" containerName="istio-proxy" containerID="cri-o://19274a3d7cf457672269ce51ccf4681eae3703c9b06eff44c95746cb7d530f23" gracePeriod=30
Oct 30 19:16:49.302849 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.302813    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"]
Oct 30 19:16:49.303082 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.303023    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl" podUID="03d301d0-7d5a-42e3-9e91-b63a3c1520fd" containerName="maas-api" containerID="cri-o://8489d8f9792c1e9930ddeeed77d87ee1953c6df1a48faa3aa9577ac952cda3c1" gracePeriod=30
Oct 30 19:16:49.444304 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.444273    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:49.697367 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.697344    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:16:49.744896 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.744863    5704 generic.go:356] "Generic (PLEG): container finished" podID="03d301d0-7d5a-42e3-9e91-b63a3c1520fd" containerID="8489d8f9792c1e9930ddeeed77d87ee1953c6df1a48faa3aa9577ac952cda3c1" exitCode=0
Oct 30 19:16:49.745049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.744908    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"
Oct 30 19:16:49.745049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.744939    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl" event={"ID":"03d301d0-7d5a-42e3-9e91-b63a3c1520fd","Type":"ContainerDied","Data":"8489d8f9792c1e9930ddeeed77d87ee1953c6df1a48faa3aa9577ac952cda3c1"}
Oct 30 19:16:49.745049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.744991    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl" event={"ID":"03d301d0-7d5a-42e3-9e91-b63a3c1520fd","Type":"ContainerDied","Data":"e7b7002e660fb7da0dd2e952413a04cec571147b04bc912ec321e95cca22325f"}
Oct 30 19:16:49.745049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.745014    5704 scope.go:122] "RemoveContainer" containerID="8489d8f9792c1e9930ddeeed77d87ee1953c6df1a48faa3aa9577ac952cda3c1"
Oct 30 19:16:49.757138 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.757112    5704 scope.go:122] "RemoveContainer" containerID="8489d8f9792c1e9930ddeeed77d87ee1953c6df1a48faa3aa9577ac952cda3c1"
Oct 30 19:16:49.757389 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:16:49.757371    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8489d8f9792c1e9930ddeeed77d87ee1953c6df1a48faa3aa9577ac952cda3c1\": container with ID starting with 8489d8f9792c1e9930ddeeed77d87ee1953c6df1a48faa3aa9577ac952cda3c1 not found: ID does not exist" containerID="8489d8f9792c1e9930ddeeed77d87ee1953c6df1a48faa3aa9577ac952cda3c1"
Oct 30 19:16:49.757438 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.757397    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8489d8f9792c1e9930ddeeed77d87ee1953c6df1a48faa3aa9577ac952cda3c1"} err="failed to get container status \"8489d8f9792c1e9930ddeeed77d87ee1953c6df1a48faa3aa9577ac952cda3c1\": rpc error: code = NotFound desc = could not find container \"8489d8f9792c1e9930ddeeed77d87ee1953c6df1a48faa3aa9577ac952cda3c1\": container with ID starting with 8489d8f9792c1e9930ddeeed77d87ee1953c6df1a48faa3aa9577ac952cda3c1 not found: ID does not exist"
Oct 30 19:16:49.807061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.803981    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-metrics-certs\") pod \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") "
Oct 30 19:16:49.807061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.804049    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-maas-api-tls\") pod \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") "
Oct 30 19:16:49.807061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.804178    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-kube-api-access-5wlvr\" (UniqueName: \"kubernetes.io/projected/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-kube-api-access-5wlvr\") pod \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") "
Oct 30 19:16:49.807061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.804410    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-openshift-service-ca\") pod \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\" (UID: \"03d301d0-7d5a-42e3-9e91-b63a3c1520fd\") "
Oct 30 19:16:49.823714 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.823677    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-metrics-certs" pod "03d301d0-7d5a-42e3-9e91-b63a3c1520fd" (UID: "03d301d0-7d5a-42e3-9e91-b63a3c1520fd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:49.829587 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.829237    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-openshift-service-ca" pod "03d301d0-7d5a-42e3-9e91-b63a3c1520fd" (UID: "03d301d0-7d5a-42e3-9e91-b63a3c1520fd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:49.860261 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.858185    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-maas-api-tls" pod "03d301d0-7d5a-42e3-9e91-b63a3c1520fd" (UID: "03d301d0-7d5a-42e3-9e91-b63a3c1520fd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:16:49.860261 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.858313    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-kube-api-access-5wlvr" pod "03d301d0-7d5a-42e3-9e91-b63a3c1520fd" (UID: "03d301d0-7d5a-42e3-9e91-b63a3c1520fd"). InnerVolumeSpecName "kube-api-access-5wlvr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:49.918057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.915274    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:49.918057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.915307    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:49.918057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.915325    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wlvr\" (UniqueName: \"kubernetes.io/projected/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-kube-api-access-5wlvr\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:49.918057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:49.915340    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03d301d0-7d5a-42e3-9e91-b63a3c1520fd-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:50.162494 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:50.162465    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"]
Oct 30 19:16:50.168122 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:50.168099    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-4f4614-84f95f85b7-f6sjl"]
Oct 30 19:16:50.543598 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:50.543566    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:50.774712 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:50.774677    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:16:50.774897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:50.774731    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r" podUID="1b169d63-af95-42fe-bcca-457d1047fcbf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:16:50.963759 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:50.963659    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:51.183472 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:51.183432    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03d301d0-7d5a-42e3-9e91-b63a3c1520fd" path="/var/lib/kubelet/pods/03d301d0-7d5a-42e3-9e91-b63a3c1520fd/volumes"
Oct 30 19:16:51.227446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:51.227279    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:51.860947 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:51.860911    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:52.167778 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:52.167694    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:52.372019 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:52.371980    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:52.557378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:52.557333    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:54.179183 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.179158    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:16:54.217184 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.217163    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:16:54.231742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.231720    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b7d9ad6f-a16b-4647-a761-a417921cca9a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d9ad6f-a16b-4647-a761-a417921cca9a-istiod-ca-cert\") pod \"b7d9ad6f-a16b-4647-a761-a417921cca9a\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") "
Oct 30 19:16:54.231847 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.231757    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-kube-api-access-kcwrg\" (UniqueName: \"kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-kube-api-access-kcwrg\") pod \"1b169d63-af95-42fe-bcca-457d1047fcbf\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") "
Oct 30 19:16:54.231847 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.231790    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-token\" (UniqueName: \"kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-token\") pod \"1b169d63-af95-42fe-bcca-457d1047fcbf\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") "
Oct 30 19:16:54.231847 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.231817    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-data\") pod \"1b169d63-af95-42fe-bcca-457d1047fcbf\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") "
Oct 30 19:16:54.231847 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.231835    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-socket\") pod \"1b169d63-af95-42fe-bcca-457d1047fcbf\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") "
Oct 30 19:16:54.232009 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.231920    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-certs\") pod \"b7d9ad6f-a16b-4647-a761-a417921cca9a\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") "
Oct 30 19:16:54.232009 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.231960    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-socket\") pod \"b7d9ad6f-a16b-4647-a761-a417921cca9a\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") "
Oct 30 19:16:54.232159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232062    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-envoy\") pod \"1b169d63-af95-42fe-bcca-457d1047fcbf\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") "
Oct 30 19:16:54.232159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232083    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7d9ad6f-a16b-4647-a761-a417921cca9a-istiod-ca-cert" pod "b7d9ad6f-a16b-4647-a761-a417921cca9a" (UID: "b7d9ad6f-a16b-4647-a761-a417921cca9a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:54.232159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232113    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-credential-socket\") pod \"b7d9ad6f-a16b-4647-a761-a417921cca9a\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") "
Oct 30 19:16:54.232159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232138    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-data" pod "1b169d63-af95-42fe-bcca-457d1047fcbf" (UID: "1b169d63-af95-42fe-bcca-457d1047fcbf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.232366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232171    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b169d63-af95-42fe-bcca-457d1047fcbf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b169d63-af95-42fe-bcca-457d1047fcbf-istiod-ca-cert\") pod \"1b169d63-af95-42fe-bcca-457d1047fcbf\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") "
Oct 30 19:16:54.232366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232225    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-envoy\") pod \"b7d9ad6f-a16b-4647-a761-a417921cca9a\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") "
Oct 30 19:16:54.232366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232304    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-kube-api-access-bgh2v\" (UniqueName: \"kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-kube-api-access-bgh2v\") pod \"b7d9ad6f-a16b-4647-a761-a417921cca9a\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") "
Oct 30 19:16:54.232366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232333    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-credential-socket\") pod \"1b169d63-af95-42fe-bcca-457d1047fcbf\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") "
Oct 30 19:16:54.232576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232364    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-certs" pod "b7d9ad6f-a16b-4647-a761-a417921cca9a" (UID: "b7d9ad6f-a16b-4647-a761-a417921cca9a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.232576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232376    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-credential-socket" pod "b7d9ad6f-a16b-4647-a761-a417921cca9a" (UID: "b7d9ad6f-a16b-4647-a761-a417921cca9a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.232576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232377    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-podinfo\") pod \"b7d9ad6f-a16b-4647-a761-a417921cca9a\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") "
Oct 30 19:16:54.232576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232388    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-socket" pod "b7d9ad6f-a16b-4647-a761-a417921cca9a" (UID: "b7d9ad6f-a16b-4647-a761-a417921cca9a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.232576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232430    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-token\" (UniqueName: \"kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-token\") pod \"b7d9ad6f-a16b-4647-a761-a417921cca9a\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") "
Oct 30 19:16:54.232576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232495    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-data\") pod \"b7d9ad6f-a16b-4647-a761-a417921cca9a\" (UID: \"b7d9ad6f-a16b-4647-a761-a417921cca9a\") "
Oct 30 19:16:54.232896 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232605    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-certs\") pod \"1b169d63-af95-42fe-bcca-457d1047fcbf\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") "
Oct 30 19:16:54.232896 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232665    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-podinfo\") pod \"1b169d63-af95-42fe-bcca-457d1047fcbf\" (UID: \"1b169d63-af95-42fe-bcca-457d1047fcbf\") "
Oct 30 19:16:54.232896 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232695    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b169d63-af95-42fe-bcca-457d1047fcbf-istiod-ca-cert" pod "1b169d63-af95-42fe-bcca-457d1047fcbf" (UID: "1b169d63-af95-42fe-bcca-457d1047fcbf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:16:54.233126 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.232922    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-data" pod "b7d9ad6f-a16b-4647-a761-a417921cca9a" (UID: "b7d9ad6f-a16b-4647-a761-a417921cca9a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.233233    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-certs" pod "1b169d63-af95-42fe-bcca-457d1047fcbf" (UID: "1b169d63-af95-42fe-bcca-457d1047fcbf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.234091    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-credential-socket" pod "1b169d63-af95-42fe-bcca-457d1047fcbf" (UID: "1b169d63-af95-42fe-bcca-457d1047fcbf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.234772    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-socket" pod "1b169d63-af95-42fe-bcca-457d1047fcbf" (UID: "1b169d63-af95-42fe-bcca-457d1047fcbf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.235471    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-token" pod "b7d9ad6f-a16b-4647-a761-a417921cca9a" (UID: "b7d9ad6f-a16b-4647-a761-a417921cca9a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.235499    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-token" pod "1b169d63-af95-42fe-bcca-457d1047fcbf" (UID: "1b169d63-af95-42fe-bcca-457d1047fcbf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.235760    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.235783    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1b169d63-af95-42fe-bcca-457d1047fcbf-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.235798    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.235814    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.235850    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.235864    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b7d9ad6f-a16b-4647-a761-a417921cca9a-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.235881    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.235907    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.236121    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.236135    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.236155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.236147    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-kube-api-access-bgh2v" pod "b7d9ad6f-a16b-4647-a761-a417921cca9a" (UID: "b7d9ad6f-a16b-4647-a761-a417921cca9a"). InnerVolumeSpecName "kube-api-access-bgh2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:54.236944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.236484    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-kube-api-access-kcwrg" pod "1b169d63-af95-42fe-bcca-457d1047fcbf" (UID: "1b169d63-af95-42fe-bcca-457d1047fcbf"). InnerVolumeSpecName "kube-api-access-kcwrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:16:54.236944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.236508    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-podinfo" pod "b7d9ad6f-a16b-4647-a761-a417921cca9a" (UID: "b7d9ad6f-a16b-4647-a761-a417921cca9a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:16:54.236944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.236624    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-envoy" pod "b7d9ad6f-a16b-4647-a761-a417921cca9a" (UID: "b7d9ad6f-a16b-4647-a761-a417921cca9a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.236944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.236794    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-envoy" pod "1b169d63-af95-42fe-bcca-457d1047fcbf" (UID: "1b169d63-af95-42fe-bcca-457d1047fcbf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:16:54.237930 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.237909    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-podinfo" pod "1b169d63-af95-42fe-bcca-457d1047fcbf" (UID: "1b169d63-af95-42fe-bcca-457d1047fcbf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:16:54.337400 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.337369    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.337538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.337405    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bgh2v\" (UniqueName: \"kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-kube-api-access-bgh2v\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.337538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.337421    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.337538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.337435    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b7d9ad6f-a16b-4647-a761-a417921cca9a-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.337538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.337448    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.337538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.337462    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcwrg\" (UniqueName: \"kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-kube-api-access-kcwrg\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.337538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.337478    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.337538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.337493    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1b169d63-af95-42fe-bcca-457d1047fcbf-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:16:54.600520 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.600487    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:16:54.780868 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.780827    5704 generic.go:356] "Generic (PLEG): container finished" podID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerID="e66308ee491db3d7ac854961c995ea804d1c6d94561d4d13e25652b84ba2b109" exitCode=0
Oct 30 19:16:54.781065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.780896    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"
Oct 30 19:16:54.781065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.780915    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" event={"ID":"b7d9ad6f-a16b-4647-a761-a417921cca9a","Type":"ContainerDied","Data":"e66308ee491db3d7ac854961c995ea804d1c6d94561d4d13e25652b84ba2b109"}
Oct 30 19:16:54.781065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.780964    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg" event={"ID":"b7d9ad6f-a16b-4647-a761-a417921cca9a","Type":"ContainerDied","Data":"1a0921d30d8e22c769862f4ee2fd6d435c68320d6fa07a1dc6423795e22c1042"}
Oct 30 19:16:54.781065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.780993    5704 scope.go:122] "RemoveContainer" containerID="e66308ee491db3d7ac854961c995ea804d1c6d94561d4d13e25652b84ba2b109"
Oct 30 19:16:54.782604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.782584    5704 generic.go:356] "Generic (PLEG): container finished" podID="1b169d63-af95-42fe-bcca-457d1047fcbf" containerID="19274a3d7cf457672269ce51ccf4681eae3703c9b06eff44c95746cb7d530f23" exitCode=0
Oct 30 19:16:54.782731 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.782705    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"
Oct 30 19:16:54.782731 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.782712    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r" event={"ID":"1b169d63-af95-42fe-bcca-457d1047fcbf","Type":"ContainerDied","Data":"19274a3d7cf457672269ce51ccf4681eae3703c9b06eff44c95746cb7d530f23"}
Oct 30 19:16:54.782848 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.782742    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r" event={"ID":"1b169d63-af95-42fe-bcca-457d1047fcbf","Type":"ContainerDied","Data":"d26595ef180a01720155cdad05bb39ff0fd4e3ec4ba7ad3cb4350777d84e3727"}
Oct 30 19:16:54.793726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.793702    5704 scope.go:122] "RemoveContainer" containerID="e66308ee491db3d7ac854961c995ea804d1c6d94561d4d13e25652b84ba2b109"
Oct 30 19:16:54.794199 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:16:54.794018    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e66308ee491db3d7ac854961c995ea804d1c6d94561d4d13e25652b84ba2b109\": container with ID starting with e66308ee491db3d7ac854961c995ea804d1c6d94561d4d13e25652b84ba2b109 not found: ID does not exist" containerID="e66308ee491db3d7ac854961c995ea804d1c6d94561d4d13e25652b84ba2b109"
Oct 30 19:16:54.794199 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.794062    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e66308ee491db3d7ac854961c995ea804d1c6d94561d4d13e25652b84ba2b109"} err="failed to get container status \"e66308ee491db3d7ac854961c995ea804d1c6d94561d4d13e25652b84ba2b109\": rpc error: code = NotFound desc = could not find container \"e66308ee491db3d7ac854961c995ea804d1c6d94561d4d13e25652b84ba2b109\": container with ID starting with e66308ee491db3d7ac854961c995ea804d1c6d94561d4d13e25652b84ba2b109 not found: ID does not exist"
Oct 30 19:16:54.794199 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.794086    5704 scope.go:122] "RemoveContainer" containerID="19274a3d7cf457672269ce51ccf4681eae3703c9b06eff44c95746cb7d530f23"
Oct 30 19:16:54.803886 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.803868    5704 scope.go:122] "RemoveContainer" containerID="19274a3d7cf457672269ce51ccf4681eae3703c9b06eff44c95746cb7d530f23"
Oct 30 19:16:54.804155 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:16:54.804138    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19274a3d7cf457672269ce51ccf4681eae3703c9b06eff44c95746cb7d530f23\": container with ID starting with 19274a3d7cf457672269ce51ccf4681eae3703c9b06eff44c95746cb7d530f23 not found: ID does not exist" containerID="19274a3d7cf457672269ce51ccf4681eae3703c9b06eff44c95746cb7d530f23"
Oct 30 19:16:54.804214 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.804162    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19274a3d7cf457672269ce51ccf4681eae3703c9b06eff44c95746cb7d530f23"} err="failed to get container status \"19274a3d7cf457672269ce51ccf4681eae3703c9b06eff44c95746cb7d530f23\": rpc error: code = NotFound desc = could not find container \"19274a3d7cf457672269ce51ccf4681eae3703c9b06eff44c95746cb7d530f23\": container with ID starting with 19274a3d7cf457672269ce51ccf4681eae3703c9b06eff44c95746cb7d530f23 not found: ID does not exist"
Oct 30 19:16:54.975057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:54.971109    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"]
Oct 30 19:16:55.027120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:55.017480    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-z2kqg"]
Oct 30 19:16:55.110349 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:55.110160    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"]
Oct 30 19:16:55.117322 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:55.116532    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-sz25r"]
Oct 30 19:16:55.181163 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:55.181080    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b169d63-af95-42fe-bcca-457d1047fcbf" path="/var/lib/kubelet/pods/1b169d63-af95-42fe-bcca-457d1047fcbf/volumes"
Oct 30 19:16:55.181771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:55.181750    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" path="/var/lib/kubelet/pods/b7d9ad6f-a16b-4647-a761-a417921cca9a/volumes"
Oct 30 19:16:57.207841 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.207813    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"]
Oct 30 19:16:57.208442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.208428    5704 cpu_manager.go:414] "containerMap: removing container" podUID="c6a1e289-e8f5-456f-8d7c-1eca2944324b" containerName="payload-pre-processing"
Oct 30 19:16:57.208512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.208443    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c6a1e289-e8f5-456f-8d7c-1eca2944324b" containerName="payload-pre-processing"
Oct 30 19:16:57.208512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.208456    5704 cpu_manager.go:414] "containerMap: removing container" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerName="istio-proxy"
Oct 30 19:16:57.208512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.208463    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerName="istio-proxy"
Oct 30 19:16:57.208512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.208474    5704 cpu_manager.go:414] "containerMap: removing container" podUID="03d301d0-7d5a-42e3-9e91-b63a3c1520fd" containerName="maas-api"
Oct 30 19:16:57.208512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.208478    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03d301d0-7d5a-42e3-9e91-b63a3c1520fd" containerName="maas-api"
Oct 30 19:16:57.208512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.208505    5704 cpu_manager.go:414] "containerMap: removing container" podUID="1b169d63-af95-42fe-bcca-457d1047fcbf" containerName="istio-proxy"
Oct 30 19:16:57.208512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.208511    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b169d63-af95-42fe-bcca-457d1047fcbf" containerName="istio-proxy"
Oct 30 19:16:57.208797 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.208609    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="c6a1e289-e8f5-456f-8d7c-1eca2944324b" containerName="payload-pre-processing"
Oct 30 19:16:57.208797 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.208621    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="b7d9ad6f-a16b-4647-a761-a417921cca9a" containerName="istio-proxy"
Oct 30 19:16:57.208797 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.208630    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="03d301d0-7d5a-42e3-9e91-b63a3c1520fd" containerName="maas-api"
Oct 30 19:16:57.208797 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.208648    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b169d63-af95-42fe-bcca-457d1047fcbf" containerName="istio-proxy"
Oct 30 19:16:57.214054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.214019    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.263736 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.263715    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-2ec7ea-serving-cert\""
Oct 30 19:16:57.314728 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.314697    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"]
Oct 30 19:16:57.324323 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.324299    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-maas-api-tls\") pod \"maas-api-e2e-api-b-2ec7ea-5569697464-blz7c\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.324446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.324348    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzvqd\" (UniqueName: \"kubernetes.io/projected/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-kube-api-access-mzvqd\") pod \"maas-api-e2e-api-b-2ec7ea-5569697464-blz7c\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.324446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.324418    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-metrics-certs\") pod \"maas-api-e2e-api-b-2ec7ea-5569697464-blz7c\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.324577 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.324474    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-openshift-service-ca\") pod \"maas-api-e2e-api-b-2ec7ea-5569697464-blz7c\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.425508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.425477    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-maas-api-tls\") pod \"maas-api-e2e-api-b-2ec7ea-5569697464-blz7c\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.425655 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.425521    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzvqd\" (UniqueName: \"kubernetes.io/projected/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-kube-api-access-mzvqd\") pod \"maas-api-e2e-api-b-2ec7ea-5569697464-blz7c\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.425655 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.425636    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-metrics-certs\") pod \"maas-api-e2e-api-b-2ec7ea-5569697464-blz7c\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.425768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.425724    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-openshift-service-ca\") pod \"maas-api-e2e-api-b-2ec7ea-5569697464-blz7c\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.426378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.426352    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-openshift-service-ca\") pod \"maas-api-e2e-api-b-2ec7ea-5569697464-blz7c\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.427976 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.427951    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-maas-api-tls\") pod \"maas-api-e2e-api-b-2ec7ea-5569697464-blz7c\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.428081 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.428064    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-metrics-certs\") pod \"maas-api-e2e-api-b-2ec7ea-5569697464-blz7c\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.439654 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.439625    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:57.482886 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.482826    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzvqd\" (UniqueName: \"kubernetes.io/projected/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-kube-api-access-mzvqd\") pod \"maas-api-e2e-api-b-2ec7ea-5569697464-blz7c\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.527448 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.527423    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:57.690070 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.680639    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:57.900432 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:57.900408    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"]
Oct 30 19:16:57.902600 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:16:57.902570    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01f17cad_ebff_4abf_b7a6_a7dcf2baa2fe.slice/crio-fd565ce2f25345802dfe538a7a8eeabf23b3ff89adc59616b2d068b7bf105dcf WatchSource:0}: Error finding container fd565ce2f25345802dfe538a7a8eeabf23b3ff89adc59616b2d068b7bf105dcf: Status 404 returned error can't find the container with id fd565ce2f25345802dfe538a7a8eeabf23b3ff89adc59616b2d068b7bf105dcf
Oct 30 19:16:58.819509 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:58.819480    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c" event={"ID":"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe","Type":"ContainerStarted","Data":"fd565ce2f25345802dfe538a7a8eeabf23b3ff89adc59616b2d068b7bf105dcf"}
Oct 30 19:16:58.852612 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:58.852584    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:59.081726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:59.081660    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:59.299056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:59.298280    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:16:59.828461 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:59.828425    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c" event={"ID":"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe","Type":"ContainerStarted","Data":"d6c8f0b413a487fd5d031e59d93ce1479285197c887a929abc0926529f79e3d3"}
Oct 30 19:16:59.828966 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:59.828504    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:16:59.942982 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:16:59.942928    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c" podStartSLOduration=2.007774609 podStartE2EDuration="2.942912556s" totalImagesPullingTime="935.137947ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:16:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:16:59.939774282 +0000 UTC m=+1792.885942186" watchObservedRunningTime="2026-10-30 19:16:59.942912556 +0000 UTC m=+1792.889080459"
Oct 30 19:17:02.402206 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:02.402170    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:02.919904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:02.919873    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"]
Oct 30 19:17:02.921971 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:02.920089    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8" podUID="430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" containerName="istio-proxy" containerID="cri-o://ec322765a72550e46a24993604b200ae4a48de875ac09569224334da7a72cbb3" gracePeriod=30
Oct 30 19:17:02.941433 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:02.941397    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"]
Oct 30 19:17:02.941597 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:02.941577    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk" podUID="29d97c69-272f-4d4d-95de-6550fdce3b3d" containerName="istio-proxy" containerID="cri-o://cf509bbb717d0dd6bea801422bc9982cba0c554e9e7627a64ff56e112c370ea8" gracePeriod=30
Oct 30 19:17:03.385062 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:17:03.385006    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:17:04.383474 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:04.383440    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:05.839882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:05.839850    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:17:07.612559 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:07.612522    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:08.211003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.210979    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:17:08.229856 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.229827    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-credential-socket\") pod \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") "
Oct 30 19:17:08.230015 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.229881    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-socket\") pod \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") "
Oct 30 19:17:08.230015 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.229997    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-certs\") pod \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") "
Oct 30 19:17:08.230189 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.230108    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istiod-ca-cert\") pod \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") "
Oct 30 19:17:08.230189 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.230162    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-data\") pod \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") "
Oct 30 19:17:08.230340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.230218    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-envoy\") pod \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") "
Oct 30 19:17:08.230340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.230274    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-token\" (UniqueName: \"kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-token\") pod \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") "
Oct 30 19:17:08.230340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.230272    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-credential-socket" pod "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" (UID: "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:08.230340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.230311    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-podinfo\") pod \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") "
Oct 30 19:17:08.230618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.230352    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-kube-api-access-9dxk4\" (UniqueName: \"kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-kube-api-access-9dxk4\") pod \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\" (UID: \"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38\") "
Oct 30 19:17:08.230618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.230578    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istiod-ca-cert" pod "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" (UID: "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:08.231098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.231076    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.231224 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.231102    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.231379 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.231359    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-socket" pod "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" (UID: "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:08.231552 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.231533    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-certs" pod "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" (UID: "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:08.233190 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.233161    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-kube-api-access-9dxk4" pod "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" (UID: "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38"). InnerVolumeSpecName "kube-api-access-9dxk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:08.234259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.234225    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-envoy" pod "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" (UID: "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:08.234422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.234405    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-data" pod "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" (UID: "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:08.235176 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.235144    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-token" pod "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" (UID: "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:08.236312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.236287    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-podinfo" pod "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" (UID: "430edd83-d8d5-4c57-b0dc-7dfc7b98ba38"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:08.242824 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.242803    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:17:08.332483 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.332459    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-certs\") pod \"29d97c69-272f-4d4d-95de-6550fdce3b3d\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") "
Oct 30 19:17:08.332593 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.332543    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-data\") pod \"29d97c69-272f-4d4d-95de-6550fdce3b3d\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") "
Oct 30 19:17:08.332593 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.332574    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-token\" (UniqueName: \"kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-token\") pod \"29d97c69-272f-4d4d-95de-6550fdce3b3d\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") "
Oct 30 19:17:08.332675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.332604    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-credential-socket\") pod \"29d97c69-272f-4d4d-95de-6550fdce3b3d\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") "
Oct 30 19:17:08.332675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.332636    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-socket\") pod \"29d97c69-272f-4d4d-95de-6550fdce3b3d\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") "
Oct 30 19:17:08.332756 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.332672    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-envoy\") pod \"29d97c69-272f-4d4d-95de-6550fdce3b3d\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") "
Oct 30 19:17:08.332756 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.332716    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-certs" pod "29d97c69-272f-4d4d-95de-6550fdce3b3d" (UID: "29d97c69-272f-4d4d-95de-6550fdce3b3d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:08.332860 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.332777    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/29d97c69-272f-4d4d-95de-6550fdce3b3d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29d97c69-272f-4d4d-95de-6550fdce3b3d-istiod-ca-cert\") pod \"29d97c69-272f-4d4d-95de-6550fdce3b3d\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") "
Oct 30 19:17:08.332860 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.332829    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-data" pod "29d97c69-272f-4d4d-95de-6550fdce3b3d" (UID: "29d97c69-272f-4d4d-95de-6550fdce3b3d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:08.332860 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.332850    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-podinfo\") pod \"29d97c69-272f-4d4d-95de-6550fdce3b3d\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") "
Oct 30 19:17:08.333016 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.332944    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-credential-socket" pod "29d97c69-272f-4d4d-95de-6550fdce3b3d" (UID: "29d97c69-272f-4d4d-95de-6550fdce3b3d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:08.333016 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.332952    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-kube-api-access-zgj4l\" (UniqueName: \"kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-kube-api-access-zgj4l\") pod \"29d97c69-272f-4d4d-95de-6550fdce3b3d\" (UID: \"29d97c69-272f-4d4d-95de-6550fdce3b3d\") "
Oct 30 19:17:08.333165 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333055    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-socket" pod "29d97c69-272f-4d4d-95de-6550fdce3b3d" (UID: "29d97c69-272f-4d4d-95de-6550fdce3b3d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:08.333235 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333186    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29d97c69-272f-4d4d-95de-6550fdce3b3d-istiod-ca-cert" pod "29d97c69-272f-4d4d-95de-6550fdce3b3d" (UID: "29d97c69-272f-4d4d-95de-6550fdce3b3d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:08.333642 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333617    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.333730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333648    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.333730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333666    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.333730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333681    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dxk4\" (UniqueName: \"kubernetes.io/projected/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-kube-api-access-9dxk4\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.333730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333697    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.333730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333711    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.333730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333725    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.334077 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333739    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.334077 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333753    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.334077 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333765    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29d97c69-272f-4d4d-95de-6550fdce3b3d-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.334077 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333778    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.334077 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.333791    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.334825 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.334798    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-token" pod "29d97c69-272f-4d4d-95de-6550fdce3b3d" (UID: "29d97c69-272f-4d4d-95de-6550fdce3b3d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:08.334907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.334876    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-kube-api-access-zgj4l" pod "29d97c69-272f-4d4d-95de-6550fdce3b3d" (UID: "29d97c69-272f-4d4d-95de-6550fdce3b3d"). InnerVolumeSpecName "kube-api-access-zgj4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:08.334907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.334884    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-podinfo" pod "29d97c69-272f-4d4d-95de-6550fdce3b3d" (UID: "29d97c69-272f-4d4d-95de-6550fdce3b3d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:08.334980 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.334949    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-envoy" pod "29d97c69-272f-4d4d-95de-6550fdce3b3d" (UID: "29d97c69-272f-4d4d-95de-6550fdce3b3d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:08.435353 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.435281    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.435353 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.435307    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.435353 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.435318    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29d97c69-272f-4d4d-95de-6550fdce3b3d-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.435353 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.435327    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zgj4l\" (UniqueName: \"kubernetes.io/projected/29d97c69-272f-4d4d-95de-6550fdce3b3d-kube-api-access-zgj4l\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:08.898443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.898409    5704 generic.go:356] "Generic (PLEG): container finished" podID="430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" containerID="ec322765a72550e46a24993604b200ae4a48de875ac09569224334da7a72cbb3" exitCode=0
Oct 30 19:17:08.898895 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.898463    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"
Oct 30 19:17:08.898895 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.898498    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8" event={"ID":"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38","Type":"ContainerDied","Data":"ec322765a72550e46a24993604b200ae4a48de875ac09569224334da7a72cbb3"}
Oct 30 19:17:08.898895 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.898570    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8" event={"ID":"430edd83-d8d5-4c57-b0dc-7dfc7b98ba38","Type":"ContainerDied","Data":"bd7d0e369a17eb9d2ff9b431b81c05611ee23a259ef54e96d1075316a9c09dbb"}
Oct 30 19:17:08.898895 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.898598    5704 scope.go:122] "RemoveContainer" containerID="ec322765a72550e46a24993604b200ae4a48de875ac09569224334da7a72cbb3"
Oct 30 19:17:08.900022 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.899995    5704 generic.go:356] "Generic (PLEG): container finished" podID="29d97c69-272f-4d4d-95de-6550fdce3b3d" containerID="cf509bbb717d0dd6bea801422bc9982cba0c554e9e7627a64ff56e112c370ea8" exitCode=0
Oct 30 19:17:08.900181 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.900028    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk" event={"ID":"29d97c69-272f-4d4d-95de-6550fdce3b3d","Type":"ContainerDied","Data":"cf509bbb717d0dd6bea801422bc9982cba0c554e9e7627a64ff56e112c370ea8"}
Oct 30 19:17:08.900181 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.900089    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"
Oct 30 19:17:08.900181 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.900104    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk" event={"ID":"29d97c69-272f-4d4d-95de-6550fdce3b3d","Type":"ContainerDied","Data":"d81ce626ef14054226beb24f8d36c3eed11e931dd36cbae890753eb12ca79fd7"}
Oct 30 19:17:08.910693 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.910549    5704 scope.go:122] "RemoveContainer" containerID="ec322765a72550e46a24993604b200ae4a48de875ac09569224334da7a72cbb3"
Oct 30 19:17:08.910902 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:17:08.910876    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec322765a72550e46a24993604b200ae4a48de875ac09569224334da7a72cbb3\": container with ID starting with ec322765a72550e46a24993604b200ae4a48de875ac09569224334da7a72cbb3 not found: ID does not exist" containerID="ec322765a72550e46a24993604b200ae4a48de875ac09569224334da7a72cbb3"
Oct 30 19:17:08.911006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.910911    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec322765a72550e46a24993604b200ae4a48de875ac09569224334da7a72cbb3"} err="failed to get container status \"ec322765a72550e46a24993604b200ae4a48de875ac09569224334da7a72cbb3\": rpc error: code = NotFound desc = could not find container \"ec322765a72550e46a24993604b200ae4a48de875ac09569224334da7a72cbb3\": container with ID starting with ec322765a72550e46a24993604b200ae4a48de875ac09569224334da7a72cbb3 not found: ID does not exist"
Oct 30 19:17:08.911006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.910933    5704 scope.go:122] "RemoveContainer" containerID="cf509bbb717d0dd6bea801422bc9982cba0c554e9e7627a64ff56e112c370ea8"
Oct 30 19:17:08.920249 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.920230    5704 scope.go:122] "RemoveContainer" containerID="cf509bbb717d0dd6bea801422bc9982cba0c554e9e7627a64ff56e112c370ea8"
Oct 30 19:17:08.920503 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:17:08.920483    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf509bbb717d0dd6bea801422bc9982cba0c554e9e7627a64ff56e112c370ea8\": container with ID starting with cf509bbb717d0dd6bea801422bc9982cba0c554e9e7627a64ff56e112c370ea8 not found: ID does not exist" containerID="cf509bbb717d0dd6bea801422bc9982cba0c554e9e7627a64ff56e112c370ea8"
Oct 30 19:17:08.920565 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:08.920509    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf509bbb717d0dd6bea801422bc9982cba0c554e9e7627a64ff56e112c370ea8"} err="failed to get container status \"cf509bbb717d0dd6bea801422bc9982cba0c554e9e7627a64ff56e112c370ea8\": rpc error: code = NotFound desc = could not find container \"cf509bbb717d0dd6bea801422bc9982cba0c554e9e7627a64ff56e112c370ea8\": container with ID starting with cf509bbb717d0dd6bea801422bc9982cba0c554e9e7627a64ff56e112c370ea8 not found: ID does not exist"
Oct 30 19:17:09.054920 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:09.054886    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"]
Oct 30 19:17:09.058733 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:09.058704    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4bnsk"]
Oct 30 19:17:09.163970 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:09.163897    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"]
Oct 30 19:17:09.171516 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:09.171495    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-4wgf8"]
Oct 30 19:17:09.183584 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:09.183562    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29d97c69-272f-4d4d-95de-6550fdce3b3d" path="/var/lib/kubelet/pods/29d97c69-272f-4d4d-95de-6550fdce3b3d/volumes"
Oct 30 19:17:09.184053 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:09.184024    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" path="/var/lib/kubelet/pods/430edd83-d8d5-4c57-b0dc-7dfc7b98ba38/volumes"
Oct 30 19:17:11.577940 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:11.577909    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"]
Oct 30 19:17:11.895868 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:11.895793    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:12.116994 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:12.116954    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:12.381323 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:12.381290    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:14.817957 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:14.817920    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:16.578312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:16.578275    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw" podUID="372bbc79-40bf-481b-be1b-be32a5a7f7ca" containerName="payload-pre-processing" containerID="cri-o://ca2bd9277d231dc187024789afdbd2b5eb43deaf3709120a9e8526bdfa1522cc" gracePeriod=25
Oct 30 19:17:16.805057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:16.792198    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:16.921473 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:16.921443    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"
Oct 30 19:17:16.997496 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:16.997464    5704 generic.go:356] "Generic (PLEG): container finished" podID="372bbc79-40bf-481b-be1b-be32a5a7f7ca" containerID="ca2bd9277d231dc187024789afdbd2b5eb43deaf3709120a9e8526bdfa1522cc" exitCode=0
Oct 30 19:17:16.997641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:16.997521    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw" event={"ID":"372bbc79-40bf-481b-be1b-be32a5a7f7ca","Type":"ContainerDied","Data":"ca2bd9277d231dc187024789afdbd2b5eb43deaf3709120a9e8526bdfa1522cc"}
Oct 30 19:17:16.997641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:16.997523    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"
Oct 30 19:17:16.997641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:16.997596    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw" event={"ID":"372bbc79-40bf-481b-be1b-be32a5a7f7ca","Type":"ContainerDied","Data":"fff59cf499868c67330d5c8569a9bb67e074e1ea73c68ee6ef6aeb802ab0609b"}
Oct 30 19:17:16.997641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:16.997617    5704 scope.go:122] "RemoveContainer" containerID="ca2bd9277d231dc187024789afdbd2b5eb43deaf3709120a9e8526bdfa1522cc"
Oct 30 19:17:17.009021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:17.009005    5704 scope.go:122] "RemoveContainer" containerID="ca2bd9277d231dc187024789afdbd2b5eb43deaf3709120a9e8526bdfa1522cc"
Oct 30 19:17:17.009356 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:17:17.009323    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca2bd9277d231dc187024789afdbd2b5eb43deaf3709120a9e8526bdfa1522cc\": container with ID starting with ca2bd9277d231dc187024789afdbd2b5eb43deaf3709120a9e8526bdfa1522cc not found: ID does not exist" containerID="ca2bd9277d231dc187024789afdbd2b5eb43deaf3709120a9e8526bdfa1522cc"
Oct 30 19:17:17.009434 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:17.009368    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca2bd9277d231dc187024789afdbd2b5eb43deaf3709120a9e8526bdfa1522cc"} err="failed to get container status \"ca2bd9277d231dc187024789afdbd2b5eb43deaf3709120a9e8526bdfa1522cc\": rpc error: code = NotFound desc = could not find container \"ca2bd9277d231dc187024789afdbd2b5eb43deaf3709120a9e8526bdfa1522cc\": container with ID starting with ca2bd9277d231dc187024789afdbd2b5eb43deaf3709120a9e8526bdfa1522cc not found: ID does not exist"
Oct 30 19:17:17.023072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:17.023021    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/372bbc79-40bf-481b-be1b-be32a5a7f7ca-kube-api-access-ts527\" (UniqueName: \"kubernetes.io/projected/372bbc79-40bf-481b-be1b-be32a5a7f7ca-kube-api-access-ts527\") pod \"372bbc79-40bf-481b-be1b-be32a5a7f7ca\" (UID: \"372bbc79-40bf-481b-be1b-be32a5a7f7ca\") "
Oct 30 19:17:17.023185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:17.023166    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/372bbc79-40bf-481b-be1b-be32a5a7f7ca-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/372bbc79-40bf-481b-be1b-be32a5a7f7ca-plugins-config-volume\") pod \"372bbc79-40bf-481b-be1b-be32a5a7f7ca\" (UID: \"372bbc79-40bf-481b-be1b-be32a5a7f7ca\") "
Oct 30 19:17:17.023510 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:17.023484    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/372bbc79-40bf-481b-be1b-be32a5a7f7ca-plugins-config-volume" pod "372bbc79-40bf-481b-be1b-be32a5a7f7ca" (UID: "372bbc79-40bf-481b-be1b-be32a5a7f7ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:17.023822 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:17.023803    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/372bbc79-40bf-481b-be1b-be32a5a7f7ca-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:17.024941 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:17.024922    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/372bbc79-40bf-481b-be1b-be32a5a7f7ca-kube-api-access-ts527" pod "372bbc79-40bf-481b-be1b-be32a5a7f7ca" (UID: "372bbc79-40bf-481b-be1b-be32a5a7f7ca"). InnerVolumeSpecName "kube-api-access-ts527". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:17.125856 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:17.125825    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ts527\" (UniqueName: \"kubernetes.io/projected/372bbc79-40bf-481b-be1b-be32a5a7f7ca-kube-api-access-ts527\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:17.454623 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:17.454588    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"]
Oct 30 19:17:17.490766 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:17.490734    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-f1b38f-6484f68675-sfdmw"]
Oct 30 19:17:17.561269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:17.561180    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:19.179923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:19.179894    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="372bbc79-40bf-481b-be1b-be32a5a7f7ca" path="/var/lib/kubelet/pods/372bbc79-40bf-481b-be1b-be32a5a7f7ca/volumes"
Oct 30 19:17:21.996794 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:21.996757    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:25.076446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:25.076399    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:25.260844 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:25.260804    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:26.969459 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:26.969417    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:27.306342 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.306308    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:27.543143 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.542938    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:27.706551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.706468    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"]
Oct 30 19:17:27.707811 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.707783    5704 cpu_manager.go:414] "containerMap: removing container" podUID="29d97c69-272f-4d4d-95de-6550fdce3b3d" containerName="istio-proxy"
Oct 30 19:17:27.707993 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.707980    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29d97c69-272f-4d4d-95de-6550fdce3b3d" containerName="istio-proxy"
Oct 30 19:17:27.708154 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.708139    5704 cpu_manager.go:414] "containerMap: removing container" podUID="372bbc79-40bf-481b-be1b-be32a5a7f7ca" containerName="payload-pre-processing"
Oct 30 19:17:27.708373 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.708226    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="372bbc79-40bf-481b-be1b-be32a5a7f7ca" containerName="payload-pre-processing"
Oct 30 19:17:27.708602 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.708586    5704 cpu_manager.go:414] "containerMap: removing container" podUID="430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" containerName="istio-proxy"
Oct 30 19:17:27.708689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.708679    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" containerName="istio-proxy"
Oct 30 19:17:27.708958 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.708944    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="29d97c69-272f-4d4d-95de-6550fdce3b3d" containerName="istio-proxy"
Oct 30 19:17:27.709079 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.709068    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="372bbc79-40bf-481b-be1b-be32a5a7f7ca" containerName="payload-pre-processing"
Oct 30 19:17:27.709167 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.709158    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="430edd83-d8d5-4c57-b0dc-7dfc7b98ba38" containerName="istio-proxy"
Oct 30 19:17:27.716490 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.716468    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"
Oct 30 19:17:27.763754 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.763724    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"]
Oct 30 19:17:27.765886 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.765861    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-942c5e85-dockercfg-rgvgg\""
Oct 30 19:17:27.806523 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.806485    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npr6g\" (UniqueName: \"kubernetes.io/projected/08aeb70b-bc30-42c0-871c-89afcd4a34e3-kube-api-access-npr6g\") pod \"payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv\" (UID: \"08aeb70b-bc30-42c0-871c-89afcd4a34e3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"
Oct 30 19:17:27.808290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.808259    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08aeb70b-bc30-42c0-871c-89afcd4a34e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv\" (UID: \"08aeb70b-bc30-42c0-871c-89afcd4a34e3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"
Oct 30 19:17:27.808692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.807097    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-942c5e85\""
Oct 30 19:17:27.967434 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.967319    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npr6g\" (UniqueName: \"kubernetes.io/projected/08aeb70b-bc30-42c0-871c-89afcd4a34e3-kube-api-access-npr6g\") pod \"payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv\" (UID: \"08aeb70b-bc30-42c0-871c-89afcd4a34e3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"
Oct 30 19:17:27.975433 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.970549    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08aeb70b-bc30-42c0-871c-89afcd4a34e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv\" (UID: \"08aeb70b-bc30-42c0-871c-89afcd4a34e3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"
Oct 30 19:17:27.975433 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:27.973607    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08aeb70b-bc30-42c0-871c-89afcd4a34e3-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv\" (UID: \"08aeb70b-bc30-42c0-871c-89afcd4a34e3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"
Oct 30 19:17:28.036399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.016241    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"]
Oct 30 19:17:28.036399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.035706    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npr6g\" (UniqueName: \"kubernetes.io/projected/08aeb70b-bc30-42c0-871c-89afcd4a34e3-kube-api-access-npr6g\") pod \"payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv\" (UID: \"08aeb70b-bc30-42c0-871c-89afcd4a34e3\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"
Oct 30 19:17:28.043000 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.042976    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"
Oct 30 19:17:28.045266 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.045235    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"
Oct 30 19:17:28.047792 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.047672    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"]
Oct 30 19:17:28.172919 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.172893    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gz9c\" (UniqueName: \"kubernetes.io/projected/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-kube-api-access-9gz9c\") pod \"payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d\" (UID: \"b2d70f4d-f3d6-400b-8eec-0d444b90b72b\") " pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"
Oct 30 19:17:28.173106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.172940    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-plugins-config-volume\") pod \"payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d\" (UID: \"b2d70f4d-f3d6-400b-8eec-0d444b90b72b\") " pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"
Oct 30 19:17:28.274892 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.274857    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gz9c\" (UniqueName: \"kubernetes.io/projected/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-kube-api-access-9gz9c\") pod \"payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d\" (UID: \"b2d70f4d-f3d6-400b-8eec-0d444b90b72b\") " pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"
Oct 30 19:17:28.275077 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.274965    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-plugins-config-volume\") pod \"payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d\" (UID: \"b2d70f4d-f3d6-400b-8eec-0d444b90b72b\") " pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"
Oct 30 19:17:28.275667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.275647    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-plugins-config-volume\") pod \"payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d\" (UID: \"b2d70f4d-f3d6-400b-8eec-0d444b90b72b\") " pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"
Oct 30 19:17:28.335189 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.335159    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gz9c\" (UniqueName: \"kubernetes.io/projected/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-kube-api-access-9gz9c\") pod \"payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d\" (UID: \"b2d70f4d-f3d6-400b-8eec-0d444b90b72b\") " pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"
Oct 30 19:17:28.404153 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.404124    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"
Oct 30 19:17:28.427593 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.427570    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"]
Oct 30 19:17:28.429569 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:17:28.429539    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08aeb70b_bc30_42c0_871c_89afcd4a34e3.slice/crio-2b2829af700f55aedbace6e61eafa58a6707a469b4af433c68a742ec275714c5 WatchSource:0}: Error finding container 2b2829af700f55aedbace6e61eafa58a6707a469b4af433c68a742ec275714c5: Status 404 returned error can't find the container with id 2b2829af700f55aedbace6e61eafa58a6707a469b4af433c68a742ec275714c5
Oct 30 19:17:28.814063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:28.814020    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"]
Oct 30 19:17:28.816381 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:17:28.816347    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2d70f4d_f3d6_400b_8eec_0d444b90b72b.slice/crio-cb5c68df7b9e6c17c37bb5e41c7e1cf4de3eaae7002c4a9ebd5c902ac3f40d1f WatchSource:0}: Error finding container cb5c68df7b9e6c17c37bb5e41c7e1cf4de3eaae7002c4a9ebd5c902ac3f40d1f: Status 404 returned error can't find the container with id cb5c68df7b9e6c17c37bb5e41c7e1cf4de3eaae7002c4a9ebd5c902ac3f40d1f
Oct 30 19:17:29.097262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:29.097192    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d" event={"ID":"b2d70f4d-f3d6-400b-8eec-0d444b90b72b","Type":"ContainerStarted","Data":"85833ff32f7069a992a303e0b9adde86bb673cd640157b18ae88fdbda5fe9537"}
Oct 30 19:17:29.097262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:29.097239    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d" event={"ID":"b2d70f4d-f3d6-400b-8eec-0d444b90b72b","Type":"ContainerStarted","Data":"cb5c68df7b9e6c17c37bb5e41c7e1cf4de3eaae7002c4a9ebd5c902ac3f40d1f"}
Oct 30 19:17:29.097726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:29.097382    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"
Oct 30 19:17:29.098931 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:29.098904    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv" event={"ID":"08aeb70b-bc30-42c0-871c-89afcd4a34e3","Type":"ContainerStarted","Data":"7c4e70876d7c6383d4d0e23f7bef829dc94f724318beaed593c4e60253457dd3"}
Oct 30 19:17:29.099063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:29.098947    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv" event={"ID":"08aeb70b-bc30-42c0-871c-89afcd4a34e3","Type":"ContainerStarted","Data":"2b2829af700f55aedbace6e61eafa58a6707a469b4af433c68a742ec275714c5"}
Oct 30 19:17:29.099063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:29.098968    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"
Oct 30 19:17:29.207906 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:29.207860    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d" podStartSLOduration=2.207847979 podStartE2EDuration="2.207847979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:29.206260859 +0000 UTC m=+1822.152428757" watchObservedRunningTime="2026-10-30 19:17:29.207847979 +0000 UTC m=+1822.154015881"
Oct 30 19:17:29.338471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:29.337898    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv" podStartSLOduration=2.337875317 podStartE2EDuration="2.337875317s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:29.334003228 +0000 UTC m=+1822.280171132" watchObservedRunningTime="2026-10-30 19:17:29.337875317 +0000 UTC m=+1822.284043221"
Oct 30 19:17:31.720140 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:31.720102    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:33.384267 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:17:33.384235    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:17:34.330983 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:34.330946    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"]
Oct 30 19:17:34.331173 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:34.331147    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72" podUID="34886f9b-18a4-4889-bef9-5b1c16f8ce2e" containerName="istio-proxy" containerID="cri-o://527d6d0c7656f7581f4018e37dc397f4b5a3c27701101c7288fee6f4dc3699b7" gracePeriod=30
Oct 30 19:17:34.376438 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:34.376411    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"]
Oct 30 19:17:34.376616 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:34.376590    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl" podUID="42b62545-91fd-4215-9c57-b1273c5471d6" containerName="istio-proxy" containerID="cri-o://e98cf09b68dbedca9917bafb8508794bd0b311300c5b0485b139763920de2300" gracePeriod=30
Oct 30 19:17:35.594270 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:35.594234    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"]
Oct 30 19:17:35.594677 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:35.594472    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7" podUID="bbba3e0b-6924-4b8a-bb5c-47e40fa39c59" containerName="maas-api" containerID="cri-o://b178a5f1ed41425cfd42fdef367e3217029d4fd7347c2e42bf5f7098bd6ceb85" gracePeriod=30
Oct 30 19:17:35.727407 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:35.727374    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:36.060098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.059220    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:17:36.146822 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.146792    5704 generic.go:356] "Generic (PLEG): container finished" podID="bbba3e0b-6924-4b8a-bb5c-47e40fa39c59" containerID="b178a5f1ed41425cfd42fdef367e3217029d4fd7347c2e42bf5f7098bd6ceb85" exitCode=0
Oct 30 19:17:36.147012 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.146838    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"
Oct 30 19:17:36.147012 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.146870    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7" event={"ID":"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59","Type":"ContainerDied","Data":"b178a5f1ed41425cfd42fdef367e3217029d4fd7347c2e42bf5f7098bd6ceb85"}
Oct 30 19:17:36.147012 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.146937    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7" event={"ID":"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59","Type":"ContainerDied","Data":"62336a8d94af2cfcef46816ab678f1f51b6acf7135741d5371edbbdb880a026f"}
Oct 30 19:17:36.147012 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.146969    5704 scope.go:122] "RemoveContainer" containerID="b178a5f1ed41425cfd42fdef367e3217029d4fd7347c2e42bf5f7098bd6ceb85"
Oct 30 19:17:36.156832 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.156809    5704 scope.go:122] "RemoveContainer" containerID="b178a5f1ed41425cfd42fdef367e3217029d4fd7347c2e42bf5f7098bd6ceb85"
Oct 30 19:17:36.157110 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:17:36.157088    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b178a5f1ed41425cfd42fdef367e3217029d4fd7347c2e42bf5f7098bd6ceb85\": container with ID starting with b178a5f1ed41425cfd42fdef367e3217029d4fd7347c2e42bf5f7098bd6ceb85 not found: ID does not exist" containerID="b178a5f1ed41425cfd42fdef367e3217029d4fd7347c2e42bf5f7098bd6ceb85"
Oct 30 19:17:36.157159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.157120    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b178a5f1ed41425cfd42fdef367e3217029d4fd7347c2e42bf5f7098bd6ceb85"} err="failed to get container status \"b178a5f1ed41425cfd42fdef367e3217029d4fd7347c2e42bf5f7098bd6ceb85\": rpc error: code = NotFound desc = could not find container \"b178a5f1ed41425cfd42fdef367e3217029d4fd7347c2e42bf5f7098bd6ceb85\": container with ID starting with b178a5f1ed41425cfd42fdef367e3217029d4fd7347c2e42bf5f7098bd6ceb85 not found: ID does not exist"
Oct 30 19:17:36.196748 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.196695    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-metrics-certs\") pod \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") "
Oct 30 19:17:36.196748 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.196742    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-kube-api-access-ktzgc\" (UniqueName: \"kubernetes.io/projected/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-kube-api-access-ktzgc\") pod \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") "
Oct 30 19:17:36.196937 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.196902    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-maas-api-tls\") pod \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") "
Oct 30 19:17:36.197176 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.197153    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-openshift-service-ca\") pod \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\" (UID: \"bbba3e0b-6924-4b8a-bb5c-47e40fa39c59\") "
Oct 30 19:17:36.197571 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.197536    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-openshift-service-ca" pod "bbba3e0b-6924-4b8a-bb5c-47e40fa39c59" (UID: "bbba3e0b-6924-4b8a-bb5c-47e40fa39c59"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:36.197777 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.197759    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:36.199330 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.199301    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-maas-api-tls" pod "bbba3e0b-6924-4b8a-bb5c-47e40fa39c59" (UID: "bbba3e0b-6924-4b8a-bb5c-47e40fa39c59"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:36.200194 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.200167    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-kube-api-access-ktzgc" pod "bbba3e0b-6924-4b8a-bb5c-47e40fa39c59" (UID: "bbba3e0b-6924-4b8a-bb5c-47e40fa39c59"). InnerVolumeSpecName "kube-api-access-ktzgc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:36.202173 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.202150    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-metrics-certs" pod "bbba3e0b-6924-4b8a-bb5c-47e40fa39c59" (UID: "bbba3e0b-6924-4b8a-bb5c-47e40fa39c59"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:17:36.298831 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.298806    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:36.298831 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.298833    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:36.298977 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.298843    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktzgc\" (UniqueName: \"kubernetes.io/projected/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59-kube-api-access-ktzgc\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:36.590929 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.590889    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"]
Oct 30 19:17:36.629055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:36.626942    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-c4ca9a-7d5fb6644c-2twb7"]
Oct 30 19:17:37.079715 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.079684    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"]
Oct 30 19:17:37.081020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.080561    5704 cpu_manager.go:414] "containerMap: removing container" podUID="bbba3e0b-6924-4b8a-bb5c-47e40fa39c59" containerName="maas-api"
Oct 30 19:17:37.081020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.080584    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bbba3e0b-6924-4b8a-bb5c-47e40fa39c59" containerName="maas-api"
Oct 30 19:17:37.081020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.080706    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="bbba3e0b-6924-4b8a-bb5c-47e40fa39c59" containerName="maas-api"
Oct 30 19:17:37.085614 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.085598    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.134208 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.134184    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-942c5e85-serving-cert\""
Oct 30 19:17:37.143465 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.143446    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"]
Oct 30 19:17:37.150654 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.150630    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/295e1b69-00f7-4a36-b4fb-43bf138919e1-openshift-service-ca\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.150759 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.150688    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-maas-api-tls\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.150867 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.150844    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-metrics-certs\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.150931 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.150895    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lfjdw\" (UniqueName: \"kubernetes.io/projected/295e1b69-00f7-4a36-b4fb-43bf138919e1-kube-api-access-lfjdw\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.180340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.180312    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bbba3e0b-6924-4b8a-bb5c-47e40fa39c59" path="/var/lib/kubelet/pods/bbba3e0b-6924-4b8a-bb5c-47e40fa39c59/volumes"
Oct 30 19:17:37.184277 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.184249    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:37.252477 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.252441    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/295e1b69-00f7-4a36-b4fb-43bf138919e1-openshift-service-ca\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.252639 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.252522    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-maas-api-tls\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.252714 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:17:37.252662    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-serving-cert: secret "maas-api-e2e-mt-942c5e85-serving-cert" not found
Oct 30 19:17:37.252714 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.252671    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-metrics-certs\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.252831 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:17:37.252719    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-maas-api-tls podName:295e1b69-00f7-4a36-b4fb-43bf138919e1 nodeName:}" failed. No retries permitted until 2026-10-30 19:17:37.752700222 +0000 UTC m=+1830.698868104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-maas-api-tls") pod "maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st" (UID: "295e1b69-00f7-4a36-b4fb-43bf138919e1") : secret "maas-api-e2e-mt-942c5e85-serving-cert" not found
Oct 30 19:17:37.252831 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.252758    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lfjdw\" (UniqueName: \"kubernetes.io/projected/295e1b69-00f7-4a36-b4fb-43bf138919e1-kube-api-access-lfjdw\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.253123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.253093    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/295e1b69-00f7-4a36-b4fb-43bf138919e1-openshift-service-ca\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.255130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.255109    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-metrics-certs\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.310602 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.310561    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lfjdw\" (UniqueName: \"kubernetes.io/projected/295e1b69-00f7-4a36-b4fb-43bf138919e1-kube-api-access-lfjdw\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.395112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.395010    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:37.581152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.580666    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:37.762335 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.762307    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-maas-api-tls\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.764647 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.764627    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-maas-api-tls\") pod \"maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:37.837781 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.837754    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:37.999022 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:37.998992    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:38.147522 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:38.147486    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:38.377013 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:38.376985    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"]
Oct 30 19:17:38.378796 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:17:38.378770    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod295e1b69_00f7_4a36_b4fb_43bf138919e1.slice/crio-6b24d3df51631a628bc174256bfac53ffa1f6732714474a44299ef912f2472c4 WatchSource:0}: Error finding container 6b24d3df51631a628bc174256bfac53ffa1f6732714474a44299ef912f2472c4: Status 404 returned error can't find the container with id 6b24d3df51631a628bc174256bfac53ffa1f6732714474a44299ef912f2472c4
Oct 30 19:17:39.180424 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.180398    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st" event={"ID":"295e1b69-00f7-4a36-b4fb-43bf138919e1","Type":"ContainerStarted","Data":"6b24d3df51631a628bc174256bfac53ffa1f6732714474a44299ef912f2472c4"}
Oct 30 19:17:39.644265 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.644237    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:17:39.693410 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.693381    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-token\" (UniqueName: \"kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-token\") pod \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") "
Oct 30 19:17:39.693599 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.693548    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istiod-ca-cert\") pod \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") "
Oct 30 19:17:39.693679 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.693606    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-kube-api-access-vt9wf\" (UniqueName: \"kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-kube-api-access-vt9wf\") pod \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") "
Oct 30 19:17:39.693809 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.693778    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-socket\") pod \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") "
Oct 30 19:17:39.693893 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.693828    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-envoy\") pod \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") "
Oct 30 19:17:39.693950 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.693916    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-podinfo\") pod \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") "
Oct 30 19:17:39.694094 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.693974    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-data\") pod \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") "
Oct 30 19:17:39.694202 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.694117    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-certs\") pod \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") "
Oct 30 19:17:39.694202 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.694147    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-socket" pod "34886f9b-18a4-4889-bef9-5b1c16f8ce2e" (UID: "34886f9b-18a4-4889-bef9-5b1c16f8ce2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.694202 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.694175    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-credential-socket\") pod \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\" (UID: \"34886f9b-18a4-4889-bef9-5b1c16f8ce2e\") "
Oct 30 19:17:39.694202 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.694175    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istiod-ca-cert" pod "34886f9b-18a4-4889-bef9-5b1c16f8ce2e" (UID: "34886f9b-18a4-4889-bef9-5b1c16f8ce2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:39.694417 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.694395    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-data" pod "34886f9b-18a4-4889-bef9-5b1c16f8ce2e" (UID: "34886f9b-18a4-4889-bef9-5b1c16f8ce2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.694474 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.694450    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-certs" pod "34886f9b-18a4-4889-bef9-5b1c16f8ce2e" (UID: "34886f9b-18a4-4889-bef9-5b1c16f8ce2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.694562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.694497    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-credential-socket" pod "34886f9b-18a4-4889-bef9-5b1c16f8ce2e" (UID: "34886f9b-18a4-4889-bef9-5b1c16f8ce2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.694949 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.694922    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.695089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.694954    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.695089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.694969    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.695089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.694984    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.695089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.694998    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.695841 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.695814    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-token" pod "34886f9b-18a4-4889-bef9-5b1c16f8ce2e" (UID: "34886f9b-18a4-4889-bef9-5b1c16f8ce2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:39.696084 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.696057    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-podinfo" pod "34886f9b-18a4-4889-bef9-5b1c16f8ce2e" (UID: "34886f9b-18a4-4889-bef9-5b1c16f8ce2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:39.696162 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.696137    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-envoy" pod "34886f9b-18a4-4889-bef9-5b1c16f8ce2e" (UID: "34886f9b-18a4-4889-bef9-5b1c16f8ce2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.696707 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.696688    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-kube-api-access-vt9wf" pod "34886f9b-18a4-4889-bef9-5b1c16f8ce2e" (UID: "34886f9b-18a4-4889-bef9-5b1c16f8ce2e"). InnerVolumeSpecName "kube-api-access-vt9wf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:39.708386 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.708368    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:17:39.795589 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.795566    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/42b62545-91fd-4215-9c57-b1273c5471d6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42b62545-91fd-4215-9c57-b1273c5471d6-istiod-ca-cert\") pod \"42b62545-91fd-4215-9c57-b1273c5471d6\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") "
Oct 30 19:17:39.795718 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.795599    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-kube-api-access-jkxjd\" (UniqueName: \"kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-kube-api-access-jkxjd\") pod \"42b62545-91fd-4215-9c57-b1273c5471d6\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") "
Oct 30 19:17:39.795718 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.795623    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-certs\") pod \"42b62545-91fd-4215-9c57-b1273c5471d6\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") "
Oct 30 19:17:39.795718 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.795654    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-istio-token\" (UniqueName: \"kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-istio-token\") pod \"42b62545-91fd-4215-9c57-b1273c5471d6\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") "
Oct 30 19:17:39.795718 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.795697    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-credential-socket\") pod \"42b62545-91fd-4215-9c57-b1273c5471d6\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") "
Oct 30 19:17:39.795927 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.795726    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-envoy\") pod \"42b62545-91fd-4215-9c57-b1273c5471d6\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") "
Oct 30 19:17:39.795927 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.795788    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-socket\") pod \"42b62545-91fd-4215-9c57-b1273c5471d6\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") "
Oct 30 19:17:39.795927 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.795838    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-data\") pod \"42b62545-91fd-4215-9c57-b1273c5471d6\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") "
Oct 30 19:17:39.795927 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.795876    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-certs" pod "42b62545-91fd-4215-9c57-b1273c5471d6" (UID: "42b62545-91fd-4215-9c57-b1273c5471d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.796167 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796065    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-credential-socket" pod "42b62545-91fd-4215-9c57-b1273c5471d6" (UID: "42b62545-91fd-4215-9c57-b1273c5471d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.796167 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796127    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/42b62545-91fd-4215-9c57-b1273c5471d6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42b62545-91fd-4215-9c57-b1273c5471d6-istio-podinfo\") pod \"42b62545-91fd-4215-9c57-b1273c5471d6\" (UID: \"42b62545-91fd-4215-9c57-b1273c5471d6\") "
Oct 30 19:17:39.796269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796254    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-data" pod "42b62545-91fd-4215-9c57-b1273c5471d6" (UID: "42b62545-91fd-4215-9c57-b1273c5471d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.796338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796308    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42b62545-91fd-4215-9c57-b1273c5471d6-istiod-ca-cert" pod "42b62545-91fd-4215-9c57-b1273c5471d6" (UID: "42b62545-91fd-4215-9c57-b1273c5471d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:17:39.796553 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796534    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-socket" pod "42b62545-91fd-4215-9c57-b1273c5471d6" (UID: "42b62545-91fd-4215-9c57-b1273c5471d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.796805 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796788    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.796902 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796806    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.796902 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796823    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.796902 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796832    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt9wf\" (UniqueName: \"kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-kube-api-access-vt9wf\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.796902 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796843    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.796902 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796856    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.796902 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796867    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34886f9b-18a4-4889-bef9-5b1c16f8ce2e-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.796902 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796880    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/42b62545-91fd-4215-9c57-b1273c5471d6-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.796902 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.796892    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.799071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.798308    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/42b62545-91fd-4215-9c57-b1273c5471d6-istio-podinfo" pod "42b62545-91fd-4215-9c57-b1273c5471d6" (UID: "42b62545-91fd-4215-9c57-b1273c5471d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:17:39.799071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.798316    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-istio-token" pod "42b62545-91fd-4215-9c57-b1273c5471d6" (UID: "42b62545-91fd-4215-9c57-b1273c5471d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:39.799071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.798771    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-envoy" pod "42b62545-91fd-4215-9c57-b1273c5471d6" (UID: "42b62545-91fd-4215-9c57-b1273c5471d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:17:39.803258 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.803236    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-kube-api-access-jkxjd" pod "42b62545-91fd-4215-9c57-b1273c5471d6" (UID: "42b62545-91fd-4215-9c57-b1273c5471d6"). InnerVolumeSpecName "kube-api-access-jkxjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:17:39.897761 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.897732    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/42b62545-91fd-4215-9c57-b1273c5471d6-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.897761 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.897759    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jkxjd\" (UniqueName: \"kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-kube-api-access-jkxjd\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.897761 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.897769    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/42b62545-91fd-4215-9c57-b1273c5471d6-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:39.898021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:39.897777    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/42b62545-91fd-4215-9c57-b1273c5471d6-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:17:40.107766 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.107696    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"
Oct 30 19:17:40.107766 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.107760    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"
Oct 30 19:17:40.292056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.291442    5704 generic.go:356] "Generic (PLEG): container finished" podID="34886f9b-18a4-4889-bef9-5b1c16f8ce2e" containerID="527d6d0c7656f7581f4018e37dc397f4b5a3c27701101c7288fee6f4dc3699b7" exitCode=0
Oct 30 19:17:40.292056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.291545    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72" event={"ID":"34886f9b-18a4-4889-bef9-5b1c16f8ce2e","Type":"ContainerDied","Data":"527d6d0c7656f7581f4018e37dc397f4b5a3c27701101c7288fee6f4dc3699b7"}
Oct 30 19:17:40.292056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.291594    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72" event={"ID":"34886f9b-18a4-4889-bef9-5b1c16f8ce2e","Type":"ContainerDied","Data":"d932746dbe7c01a2e5fb842b10b30fc9dd0662cd9aedbf72bdcb91d82fd27f8c"}
Oct 30 19:17:40.292056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.291624    5704 scope.go:122] "RemoveContainer" containerID="527d6d0c7656f7581f4018e37dc397f4b5a3c27701101c7288fee6f4dc3699b7"
Oct 30 19:17:40.292056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.291775    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"
Oct 30 19:17:40.303660 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.297727    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st" event={"ID":"295e1b69-00f7-4a36-b4fb-43bf138919e1","Type":"ContainerStarted","Data":"8a5feb2db4298feba8b90d4ca81e9d144b27c1a7d0b4fdcd4926fda45538c871"}
Oct 30 19:17:40.303660 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.299311    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:40.303660 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.301321    5704 generic.go:356] "Generic (PLEG): container finished" podID="42b62545-91fd-4215-9c57-b1273c5471d6" containerID="e98cf09b68dbedca9917bafb8508794bd0b311300c5b0485b139763920de2300" exitCode=0
Oct 30 19:17:40.303660 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.301372    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl" event={"ID":"42b62545-91fd-4215-9c57-b1273c5471d6","Type":"ContainerDied","Data":"e98cf09b68dbedca9917bafb8508794bd0b311300c5b0485b139763920de2300"}
Oct 30 19:17:40.303660 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.301413    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl" event={"ID":"42b62545-91fd-4215-9c57-b1273c5471d6","Type":"ContainerDied","Data":"8b88ae5024f0984985326058ace44ce2301e8a0705a00444dba99434c4a77023"}
Oct 30 19:17:40.303660 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.301487    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"
Oct 30 19:17:40.325136 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.324813    5704 scope.go:122] "RemoveContainer" containerID="527d6d0c7656f7581f4018e37dc397f4b5a3c27701101c7288fee6f4dc3699b7"
Oct 30 19:17:40.326239 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:17:40.326213    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"527d6d0c7656f7581f4018e37dc397f4b5a3c27701101c7288fee6f4dc3699b7\": container with ID starting with 527d6d0c7656f7581f4018e37dc397f4b5a3c27701101c7288fee6f4dc3699b7 not found: ID does not exist" containerID="527d6d0c7656f7581f4018e37dc397f4b5a3c27701101c7288fee6f4dc3699b7"
Oct 30 19:17:40.326504 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.326478    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"527d6d0c7656f7581f4018e37dc397f4b5a3c27701101c7288fee6f4dc3699b7"} err="failed to get container status \"527d6d0c7656f7581f4018e37dc397f4b5a3c27701101c7288fee6f4dc3699b7\": rpc error: code = NotFound desc = could not find container \"527d6d0c7656f7581f4018e37dc397f4b5a3c27701101c7288fee6f4dc3699b7\": container with ID starting with 527d6d0c7656f7581f4018e37dc397f4b5a3c27701101c7288fee6f4dc3699b7 not found: ID does not exist"
Oct 30 19:17:40.326615 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.326507    5704 scope.go:122] "RemoveContainer" containerID="e98cf09b68dbedca9917bafb8508794bd0b311300c5b0485b139763920de2300"
Oct 30 19:17:40.352658 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.352341    5704 scope.go:122] "RemoveContainer" containerID="e98cf09b68dbedca9917bafb8508794bd0b311300c5b0485b139763920de2300"
Oct 30 19:17:40.353562 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:17:40.352914    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e98cf09b68dbedca9917bafb8508794bd0b311300c5b0485b139763920de2300\": container with ID starting with e98cf09b68dbedca9917bafb8508794bd0b311300c5b0485b139763920de2300 not found: ID does not exist" containerID="e98cf09b68dbedca9917bafb8508794bd0b311300c5b0485b139763920de2300"
Oct 30 19:17:40.353562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.352952    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e98cf09b68dbedca9917bafb8508794bd0b311300c5b0485b139763920de2300"} err="failed to get container status \"e98cf09b68dbedca9917bafb8508794bd0b311300c5b0485b139763920de2300\": rpc error: code = NotFound desc = could not find container \"e98cf09b68dbedca9917bafb8508794bd0b311300c5b0485b139763920de2300\": container with ID starting with e98cf09b68dbedca9917bafb8508794bd0b311300c5b0485b139763920de2300 not found: ID does not exist"
Oct 30 19:17:40.490066 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.490021    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"]
Oct 30 19:17:40.507301 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.507271    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-6ps72"]
Oct 30 19:17:40.617552 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.617500    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st" podStartSLOduration=2.828708584 podStartE2EDuration="3.617485759s" totalImagesPullingTime="788.777175ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:17:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:17:40.615009462 +0000 UTC m=+1833.561177366" watchObservedRunningTime="2026-10-30 19:17:40.617485759 +0000 UTC m=+1833.563653661"
Oct 30 19:17:40.724930 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.724852    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"]
Oct 30 19:17:40.739301 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:40.733286    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-lf9wl"]
Oct 30 19:17:41.182121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:41.182093    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="34886f9b-18a4-4889-bef9-5b1c16f8ce2e" path="/var/lib/kubelet/pods/34886f9b-18a4-4889-bef9-5b1c16f8ce2e/volumes"
Oct 30 19:17:41.182584 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:41.182562    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="42b62545-91fd-4215-9c57-b1273c5471d6" path="/var/lib/kubelet/pods/42b62545-91fd-4215-9c57-b1273c5471d6/volumes"
Oct 30 19:17:42.399338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:42.399303    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:45.301420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:45.301394    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:45.477416 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:45.477376    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:45.693870 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:45.693789    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:46.828631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:46.828561    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:17:47.321534 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:47.321508    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:17:47.756319 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:47.756280    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:53.114897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:53.114857    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:53.641350 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:53.641313    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:56.987198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:56.987162    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"]
Oct 30 19:17:57.485718 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:57.485688    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:57.719140 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:57.719109    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:17:58.260350 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:17:58.260309    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:01.987931 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:01.987878    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl" podUID="be13b502-20cc-4754-ab38-71eee24dca18" containerName="payload-pre-processing" containerID="cri-o://66b2d023282ca244a3ea84be2c30f3571516332b33e0b3efb7e023593d21de73" gracePeriod=25
Oct 30 19:18:02.260442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.260417    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"
Oct 30 19:18:02.313022 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.312992    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/be13b502-20cc-4754-ab38-71eee24dca18-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be13b502-20cc-4754-ab38-71eee24dca18-plugins-config-volume\") pod \"be13b502-20cc-4754-ab38-71eee24dca18\" (UID: \"be13b502-20cc-4754-ab38-71eee24dca18\") "
Oct 30 19:18:02.313240 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.313130    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/be13b502-20cc-4754-ab38-71eee24dca18-kube-api-access-wzdd6\" (UniqueName: \"kubernetes.io/projected/be13b502-20cc-4754-ab38-71eee24dca18-kube-api-access-wzdd6\") pod \"be13b502-20cc-4754-ab38-71eee24dca18\" (UID: \"be13b502-20cc-4754-ab38-71eee24dca18\") "
Oct 30 19:18:02.313425 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.313401    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be13b502-20cc-4754-ab38-71eee24dca18-plugins-config-volume" pod "be13b502-20cc-4754-ab38-71eee24dca18" (UID: "be13b502-20cc-4754-ab38-71eee24dca18"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:02.313766 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.313743    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/be13b502-20cc-4754-ab38-71eee24dca18-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:02.315499 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.315467    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be13b502-20cc-4754-ab38-71eee24dca18-kube-api-access-wzdd6" pod "be13b502-20cc-4754-ab38-71eee24dca18" (UID: "be13b502-20cc-4754-ab38-71eee24dca18"). InnerVolumeSpecName "kube-api-access-wzdd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:02.414814 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.414785    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wzdd6\" (UniqueName: \"kubernetes.io/projected/be13b502-20cc-4754-ab38-71eee24dca18-kube-api-access-wzdd6\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:02.587306 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.587271    5704 generic.go:356] "Generic (PLEG): container finished" podID="be13b502-20cc-4754-ab38-71eee24dca18" containerID="66b2d023282ca244a3ea84be2c30f3571516332b33e0b3efb7e023593d21de73" exitCode=0
Oct 30 19:18:02.587486 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.587322    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"
Oct 30 19:18:02.587486 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.587350    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl" event={"ID":"be13b502-20cc-4754-ab38-71eee24dca18","Type":"ContainerDied","Data":"66b2d023282ca244a3ea84be2c30f3571516332b33e0b3efb7e023593d21de73"}
Oct 30 19:18:02.587486 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.587404    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl" event={"ID":"be13b502-20cc-4754-ab38-71eee24dca18","Type":"ContainerDied","Data":"86b4af309d61ed0c2b9e8aad8d0f98a499e02b1a191e5b2bedb90bb6e01e11a9"}
Oct 30 19:18:02.587486 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.587425    5704 scope.go:122] "RemoveContainer" containerID="66b2d023282ca244a3ea84be2c30f3571516332b33e0b3efb7e023593d21de73"
Oct 30 19:18:02.597819 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.597802    5704 scope.go:122] "RemoveContainer" containerID="66b2d023282ca244a3ea84be2c30f3571516332b33e0b3efb7e023593d21de73"
Oct 30 19:18:02.598175 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:02.598156    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66b2d023282ca244a3ea84be2c30f3571516332b33e0b3efb7e023593d21de73\": container with ID starting with 66b2d023282ca244a3ea84be2c30f3571516332b33e0b3efb7e023593d21de73 not found: ID does not exist" containerID="66b2d023282ca244a3ea84be2c30f3571516332b33e0b3efb7e023593d21de73"
Oct 30 19:18:02.598246 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.598183    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66b2d023282ca244a3ea84be2c30f3571516332b33e0b3efb7e023593d21de73"} err="failed to get container status \"66b2d023282ca244a3ea84be2c30f3571516332b33e0b3efb7e023593d21de73\": rpc error: code = NotFound desc = could not find container \"66b2d023282ca244a3ea84be2c30f3571516332b33e0b3efb7e023593d21de73\": container with ID starting with 66b2d023282ca244a3ea84be2c30f3571516332b33e0b3efb7e023593d21de73 not found: ID does not exist"
Oct 30 19:18:02.742790 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.742763    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"]
Oct 30 19:18:02.746984 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:02.746960    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-0d2ef6-bc8b6bd7-7nnkl"]
Oct 30 19:18:03.086631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:03.086599    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:03.181289 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:03.181258    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="be13b502-20cc-4754-ab38-71eee24dca18" path="/var/lib/kubelet/pods/be13b502-20cc-4754-ab38-71eee24dca18/volumes"
Oct 30 19:18:03.383817 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:03.383731    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:18:04.723571 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:04.723541    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:06.326306 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:06.326273    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"]
Oct 30 19:18:06.429447 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:06.429416    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"]
Oct 30 19:18:06.707730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:06.707639    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:06.942537 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:06.942502    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:07.041353 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.041324    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"]
Oct 30 19:18:07.041601 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.041514    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v" podUID="1324e64f-d43b-46dc-bcbf-7d5d4565b3df" containerName="maas-api" containerID="cri-o://390629f3a8f08e14964954b4b1073279e6c03bb431d5f21f1279bbc8d289d956" gracePeriod=30
Oct 30 19:18:07.175951 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.175909    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:07.390503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.390473    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:18:07.512475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.512369    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-openshift-service-ca\") pod \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") "
Oct 30 19:18:07.515110 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.512662    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-metrics-certs\" (UniqueName: \"kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-metrics-certs\") pod \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") "
Oct 30 19:18:07.515110 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.512766    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-maas-api-tls\") pod \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") "
Oct 30 19:18:07.515110 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.512806    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-kube-api-access-64c86\" (UniqueName: \"kubernetes.io/projected/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-kube-api-access-64c86\") pod \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\" (UID: \"1324e64f-d43b-46dc-bcbf-7d5d4565b3df\") "
Oct 30 19:18:07.531386 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.531351    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-openshift-service-ca" pod "1324e64f-d43b-46dc-bcbf-7d5d4565b3df" (UID: "1324e64f-d43b-46dc-bcbf-7d5d4565b3df"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:07.531637 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.531617    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-metrics-certs" pod "1324e64f-d43b-46dc-bcbf-7d5d4565b3df" (UID: "1324e64f-d43b-46dc-bcbf-7d5d4565b3df"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:18:07.555414 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.544585    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-kube-api-access-64c86" pod "1324e64f-d43b-46dc-bcbf-7d5d4565b3df" (UID: "1324e64f-d43b-46dc-bcbf-7d5d4565b3df"). InnerVolumeSpecName "kube-api-access-64c86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:07.561323 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.560802    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-maas-api-tls" pod "1324e64f-d43b-46dc-bcbf-7d5d4565b3df" (UID: "1324e64f-d43b-46dc-bcbf-7d5d4565b3df"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:18:07.614376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.614319    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:07.614376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.614343    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64c86\" (UniqueName: \"kubernetes.io/projected/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-kube-api-access-64c86\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:07.614376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.614356    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:07.614376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.614366    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1324e64f-d43b-46dc-bcbf-7d5d4565b3df-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:07.623093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.623066    5704 generic.go:356] "Generic (PLEG): container finished" podID="1324e64f-d43b-46dc-bcbf-7d5d4565b3df" containerID="390629f3a8f08e14964954b4b1073279e6c03bb431d5f21f1279bbc8d289d956" exitCode=0
Oct 30 19:18:07.623207 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.623114    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"
Oct 30 19:18:07.623207 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.623144    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v" event={"ID":"1324e64f-d43b-46dc-bcbf-7d5d4565b3df","Type":"ContainerDied","Data":"390629f3a8f08e14964954b4b1073279e6c03bb431d5f21f1279bbc8d289d956"}
Oct 30 19:18:07.623329 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.623206    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v" event={"ID":"1324e64f-d43b-46dc-bcbf-7d5d4565b3df","Type":"ContainerDied","Data":"671aabdd69a51296eaf07df0c41971af34d6ff113507d1ddf3608f331b62d1f9"}
Oct 30 19:18:07.623329 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.623238    5704 scope.go:122] "RemoveContainer" containerID="390629f3a8f08e14964954b4b1073279e6c03bb431d5f21f1279bbc8d289d956"
Oct 30 19:18:07.633247 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.633232    5704 scope.go:122] "RemoveContainer" containerID="390629f3a8f08e14964954b4b1073279e6c03bb431d5f21f1279bbc8d289d956"
Oct 30 19:18:07.633503 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:07.633485    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"390629f3a8f08e14964954b4b1073279e6c03bb431d5f21f1279bbc8d289d956\": container with ID starting with 390629f3a8f08e14964954b4b1073279e6c03bb431d5f21f1279bbc8d289d956 not found: ID does not exist" containerID="390629f3a8f08e14964954b4b1073279e6c03bb431d5f21f1279bbc8d289d956"
Oct 30 19:18:07.633551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.633510    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"390629f3a8f08e14964954b4b1073279e6c03bb431d5f21f1279bbc8d289d956"} err="failed to get container status \"390629f3a8f08e14964954b4b1073279e6c03bb431d5f21f1279bbc8d289d956\": rpc error: code = NotFound desc = could not find container \"390629f3a8f08e14964954b4b1073279e6c03bb431d5f21f1279bbc8d289d956\": container with ID starting with 390629f3a8f08e14964954b4b1073279e6c03bb431d5f21f1279bbc8d289d956 not found: ID does not exist"
Oct 30 19:18:07.784061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.783221    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"]
Oct 30 19:18:07.796055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:07.788310    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-f1b38f-98bc99dd9-2bb7v"]
Oct 30 19:18:08.654799 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:08.654766    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:08.904068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:08.904021    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:09.096886 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:09.096846    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:09.180818 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:09.180779    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1324e64f-d43b-46dc-bcbf-7d5d4565b3df" path="/var/lib/kubelet/pods/1324e64f-d43b-46dc-bcbf-7d5d4565b3df/volumes"
Oct 30 19:18:09.545674 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:09.545637    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:11.327050 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.326996    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp" podUID="2a762f75-0170-4a91-bcf4-bb2af0079517" containerName="payload-processing" containerID="cri-o://e7a19155d6ba03a402f4ebee49daa76279b06131513998f8cf2103d3549416db" gracePeriod=25
Oct 30 19:18:11.429888 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.429852    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l" podUID="fe2b1ef6-71e9-470a-ae30-aa957f4c84b6" containerName="payload-pre-processing" containerID="cri-o://6034faac46dbb2e19786f3c935d9c7fcc1e5ad2585320a88ad1d03fd13223356" gracePeriod=25
Oct 30 19:18:11.628860 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.628830    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"
Oct 30 19:18:11.652184 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.652153    5704 generic.go:356] "Generic (PLEG): container finished" podID="fe2b1ef6-71e9-470a-ae30-aa957f4c84b6" containerID="6034faac46dbb2e19786f3c935d9c7fcc1e5ad2585320a88ad1d03fd13223356" exitCode=0
Oct 30 19:18:11.652341 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.652228    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l" event={"ID":"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6","Type":"ContainerDied","Data":"6034faac46dbb2e19786f3c935d9c7fcc1e5ad2585320a88ad1d03fd13223356"}
Oct 30 19:18:11.653828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.653803    5704 generic.go:356] "Generic (PLEG): container finished" podID="2a762f75-0170-4a91-bcf4-bb2af0079517" containerID="e7a19155d6ba03a402f4ebee49daa76279b06131513998f8cf2103d3549416db" exitCode=0
Oct 30 19:18:11.653960 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.653841    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"
Oct 30 19:18:11.653960 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.653877    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp" event={"ID":"2a762f75-0170-4a91-bcf4-bb2af0079517","Type":"ContainerDied","Data":"e7a19155d6ba03a402f4ebee49daa76279b06131513998f8cf2103d3549416db"}
Oct 30 19:18:11.653960 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.653933    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp" event={"ID":"2a762f75-0170-4a91-bcf4-bb2af0079517","Type":"ContainerDied","Data":"7fa608169472c9f108baab13e78ebfd84ac8129f8da0158b9b68148bea1ec35b"}
Oct 30 19:18:11.654146 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.653965    5704 scope.go:122] "RemoveContainer" containerID="e7a19155d6ba03a402f4ebee49daa76279b06131513998f8cf2103d3549416db"
Oct 30 19:18:11.663556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.663538    5704 scope.go:122] "RemoveContainer" containerID="e7a19155d6ba03a402f4ebee49daa76279b06131513998f8cf2103d3549416db"
Oct 30 19:18:11.663846 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:11.663821    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7a19155d6ba03a402f4ebee49daa76279b06131513998f8cf2103d3549416db\": container with ID starting with e7a19155d6ba03a402f4ebee49daa76279b06131513998f8cf2103d3549416db not found: ID does not exist" containerID="e7a19155d6ba03a402f4ebee49daa76279b06131513998f8cf2103d3549416db"
Oct 30 19:18:11.663962 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.663855    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7a19155d6ba03a402f4ebee49daa76279b06131513998f8cf2103d3549416db"} err="failed to get container status \"e7a19155d6ba03a402f4ebee49daa76279b06131513998f8cf2103d3549416db\": rpc error: code = NotFound desc = could not find container \"e7a19155d6ba03a402f4ebee49daa76279b06131513998f8cf2103d3549416db\": container with ID starting with e7a19155d6ba03a402f4ebee49daa76279b06131513998f8cf2103d3549416db not found: ID does not exist"
Oct 30 19:18:11.693765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.693745    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"
Oct 30 19:18:11.785960 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.785934    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-plugins-config-volume\") pod \"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6\" (UID: \"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6\") "
Oct 30 19:18:11.786152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.786013    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a762f75-0170-4a91-bcf4-bb2af0079517-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a762f75-0170-4a91-bcf4-bb2af0079517-plugins-config-volume\") pod \"2a762f75-0170-4a91-bcf4-bb2af0079517\" (UID: \"2a762f75-0170-4a91-bcf4-bb2af0079517\") "
Oct 30 19:18:11.786152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.786081    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a762f75-0170-4a91-bcf4-bb2af0079517-kube-api-access-q7nzn\" (UniqueName: \"kubernetes.io/projected/2a762f75-0170-4a91-bcf4-bb2af0079517-kube-api-access-q7nzn\") pod \"2a762f75-0170-4a91-bcf4-bb2af0079517\" (UID: \"2a762f75-0170-4a91-bcf4-bb2af0079517\") "
Oct 30 19:18:11.786152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.786132    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-kube-api-access-klt6w\" (UniqueName: \"kubernetes.io/projected/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-kube-api-access-klt6w\") pod \"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6\" (UID: \"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6\") "
Oct 30 19:18:11.786399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.786374    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-plugins-config-volume" pod "fe2b1ef6-71e9-470a-ae30-aa957f4c84b6" (UID: "fe2b1ef6-71e9-470a-ae30-aa957f4c84b6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:11.786692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.786673    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:11.786763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.786722    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a762f75-0170-4a91-bcf4-bb2af0079517-plugins-config-volume" pod "2a762f75-0170-4a91-bcf4-bb2af0079517" (UID: "2a762f75-0170-4a91-bcf4-bb2af0079517"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:11.788525 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.788504    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-kube-api-access-klt6w" pod "fe2b1ef6-71e9-470a-ae30-aa957f4c84b6" (UID: "fe2b1ef6-71e9-470a-ae30-aa957f4c84b6"). InnerVolumeSpecName "kube-api-access-klt6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:11.788623 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.788549    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a762f75-0170-4a91-bcf4-bb2af0079517-kube-api-access-q7nzn" pod "2a762f75-0170-4a91-bcf4-bb2af0079517" (UID: "2a762f75-0170-4a91-bcf4-bb2af0079517"). InnerVolumeSpecName "kube-api-access-q7nzn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:11.887691 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.887619    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2a762f75-0170-4a91-bcf4-bb2af0079517-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:11.887691 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.887653    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q7nzn\" (UniqueName: \"kubernetes.io/projected/2a762f75-0170-4a91-bcf4-bb2af0079517-kube-api-access-q7nzn\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:11.887691 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:11.887667    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klt6w\" (UniqueName: \"kubernetes.io/projected/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6-kube-api-access-klt6w\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:12.068309 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:12.068280    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"]
Oct 30 19:18:12.072801 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:12.072775    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-f44636-6f6dbd8fc6-tdlnp"]
Oct 30 19:18:12.661567 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:12.661490    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"
Oct 30 19:18:12.661567 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:12.661498    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l" event={"ID":"fe2b1ef6-71e9-470a-ae30-aa957f4c84b6","Type":"ContainerDied","Data":"0bde6152a95abb995729fcefadf5d66f58a94de925325e22655e78fb7a36e306"}
Oct 30 19:18:12.661567 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:12.661550    5704 scope.go:122] "RemoveContainer" containerID="6034faac46dbb2e19786f3c935d9c7fcc1e5ad2585320a88ad1d03fd13223356"
Oct 30 19:18:12.817351 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:12.817321    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"]
Oct 30 19:18:12.862976 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:12.862951    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-f44636-665c87d4ff-wdj6l"]
Oct 30 19:18:13.179829 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:13.179800    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a762f75-0170-4a91-bcf4-bb2af0079517" path="/var/lib/kubelet/pods/2a762f75-0170-4a91-bcf4-bb2af0079517/volumes"
Oct 30 19:18:13.180237 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:13.180223    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fe2b1ef6-71e9-470a-ae30-aa957f4c84b6" path="/var/lib/kubelet/pods/fe2b1ef6-71e9-470a-ae30-aa957f4c84b6/volumes"
Oct 30 19:18:13.692159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:13.692127    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:15.562845 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:15.562816    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:15.739216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:15.739184    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:15.919305 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:15.919207    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:16.907399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:16.907362    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"]
Oct 30 19:18:16.907907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:16.907622    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx" podUID="a4687e07-e7bd-4a45-a89e-b7550ec67f95" containerName="istio-proxy" containerID="cri-o://da33beb1ece53ce055e4b336ea7dc797298a21eec23816da07ace3f83af73a30" gracePeriod=30
Oct 30 19:18:17.410319 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:17.410292    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"]
Oct 30 19:18:17.410518 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:17.410498    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" podUID="28fe6291-de37-42c5-abe2-76e43b0f89b3" containerName="istio-proxy" containerID="cri-o://f8ceadf9b2dc93e16c6422873275cf971130a5c32e02113f66377fb3c991f4b9" gracePeriod=30
Oct 30 19:18:17.415203 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:17.415170    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"]
Oct 30 19:18:17.415363 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:17.415344    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerName="istio-proxy" containerID="cri-o://82f89b20be049ce7a1db619a6dc4f0281895300c03aa97766acaa4bc34c62ab9" gracePeriod=30
Oct 30 19:18:17.456464 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:17.456437    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"]
Oct 30 19:18:17.456632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:17.456613    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562" podUID="307212d3-c2b4-4bc6-908e-607728b8e195" containerName="istio-proxy" containerID="cri-o://c66ace058f3d7733b1569364c72118482848894dafdce04193369e979c66fbab" gracePeriod=30
Oct 30 19:18:17.459546 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:17.459523    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"]
Oct 30 19:18:17.459733 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:17.459713    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" podUID="f78f6217-90d4-405c-bed9-36b9e9911b7c" containerName="istio-proxy" containerID="cri-o://019fdd8c5e1d451d6489b46540b16af971713bb0002d517724577dabcc3b5622" gracePeriod=30
Oct 30 19:18:18.379916 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:18.379875    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:21.655472 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:21.655433    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:18:21.655911 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:21.655504    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562" podUID="307212d3-c2b4-4bc6-908e-607728b8e195" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:18:22.186426 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.186367    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:18:22.290361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.288727    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-certs\") pod \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") "
Oct 30 19:18:22.290361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.288830    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-podinfo\") pod \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") "
Oct 30 19:18:22.290361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.288916    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-envoy\") pod \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") "
Oct 30 19:18:22.290361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.288948    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-data\") pod \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") "
Oct 30 19:18:22.290361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.289016    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-token\" (UniqueName: \"kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-token\") pod \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") "
Oct 30 19:18:22.290361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.289106    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istiod-ca-cert\") pod \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") "
Oct 30 19:18:22.290361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.289162    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-credential-socket\") pod \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") "
Oct 30 19:18:22.290361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.289227    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-kube-api-access-c2k86\" (UniqueName: \"kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-kube-api-access-c2k86\") pod \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") "
Oct 30 19:18:22.290361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.289383    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-socket\") pod \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\" (UID: \"a4687e07-e7bd-4a45-a89e-b7550ec67f95\") "
Oct 30 19:18:22.295072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.291022    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-socket" pod "a4687e07-e7bd-4a45-a89e-b7550ec67f95" (UID: "a4687e07-e7bd-4a45-a89e-b7550ec67f95"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.295702 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.295347    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-certs" pod "a4687e07-e7bd-4a45-a89e-b7550ec67f95" (UID: "a4687e07-e7bd-4a45-a89e-b7550ec67f95"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.301539 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.301345    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-credential-socket" pod "a4687e07-e7bd-4a45-a89e-b7550ec67f95" (UID: "a4687e07-e7bd-4a45-a89e-b7550ec67f95"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.302125 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.301810    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istiod-ca-cert" pod "a4687e07-e7bd-4a45-a89e-b7550ec67f95" (UID: "a4687e07-e7bd-4a45-a89e-b7550ec67f95"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:22.303955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.303931    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-data" pod "a4687e07-e7bd-4a45-a89e-b7550ec67f95" (UID: "a4687e07-e7bd-4a45-a89e-b7550ec67f95"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.366108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.364963    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-envoy" pod "a4687e07-e7bd-4a45-a89e-b7550ec67f95" (UID: "a4687e07-e7bd-4a45-a89e-b7550ec67f95"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.366108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.365692    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-kube-api-access-c2k86" pod "a4687e07-e7bd-4a45-a89e-b7550ec67f95" (UID: "a4687e07-e7bd-4a45-a89e-b7550ec67f95"). InnerVolumeSpecName "kube-api-access-c2k86". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:22.370544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.370494    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-token" pod "a4687e07-e7bd-4a45-a89e-b7550ec67f95" (UID: "a4687e07-e7bd-4a45-a89e-b7550ec67f95"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:22.376268 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.376220    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-podinfo" pod "a4687e07-e7bd-4a45-a89e-b7550ec67f95" (UID: "a4687e07-e7bd-4a45-a89e-b7550ec67f95"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:18:22.392879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.392464    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.392879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.392506    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.392879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.392520    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.392879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.392533    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.392879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.392548    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.392879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.392563    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c2k86\" (UniqueName: \"kubernetes.io/projected/a4687e07-e7bd-4a45-a89e-b7550ec67f95-kube-api-access-c2k86\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.392879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.392579    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.392879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.392591    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a4687e07-e7bd-4a45-a89e-b7550ec67f95-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.392879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.392604    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a4687e07-e7bd-4a45-a89e-b7550ec67f95-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.737125 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.736991    5704 generic.go:356] "Generic (PLEG): container finished" podID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerID="82f89b20be049ce7a1db619a6dc4f0281895300c03aa97766acaa4bc34c62ab9" exitCode=0
Oct 30 19:18:22.737125 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.737088    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" event={"ID":"eb6977e8-1680-449b-ad3e-3434eac1e0e9","Type":"ContainerDied","Data":"82f89b20be049ce7a1db619a6dc4f0281895300c03aa97766acaa4bc34c62ab9"}
Oct 30 19:18:22.738632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.738609    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:18:22.738757 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.738699    5704 generic.go:356] "Generic (PLEG): container finished" podID="28fe6291-de37-42c5-abe2-76e43b0f89b3" containerID="f8ceadf9b2dc93e16c6422873275cf971130a5c32e02113f66377fb3c991f4b9" exitCode=0
Oct 30 19:18:22.738843 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.738817    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" event={"ID":"28fe6291-de37-42c5-abe2-76e43b0f89b3","Type":"ContainerDied","Data":"f8ceadf9b2dc93e16c6422873275cf971130a5c32e02113f66377fb3c991f4b9"}
Oct 30 19:18:22.738939 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.738893    5704 scope.go:122] "RemoveContainer" containerID="f8ceadf9b2dc93e16c6422873275cf971130a5c32e02113f66377fb3c991f4b9"
Oct 30 19:18:22.740790 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.740758    5704 generic.go:356] "Generic (PLEG): container finished" podID="307212d3-c2b4-4bc6-908e-607728b8e195" containerID="c66ace058f3d7733b1569364c72118482848894dafdce04193369e979c66fbab" exitCode=0
Oct 30 19:18:22.740900 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.740828    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562" event={"ID":"307212d3-c2b4-4bc6-908e-607728b8e195","Type":"ContainerDied","Data":"c66ace058f3d7733b1569364c72118482848894dafdce04193369e979c66fbab"}
Oct 30 19:18:22.743450 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.743062    5704 generic.go:356] "Generic (PLEG): container finished" podID="a4687e07-e7bd-4a45-a89e-b7550ec67f95" containerID="da33beb1ece53ce055e4b336ea7dc797298a21eec23816da07ace3f83af73a30" exitCode=0
Oct 30 19:18:22.743450 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.743387    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"
Oct 30 19:18:22.745820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.745481    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx" event={"ID":"a4687e07-e7bd-4a45-a89e-b7550ec67f95","Type":"ContainerDied","Data":"da33beb1ece53ce055e4b336ea7dc797298a21eec23816da07ace3f83af73a30"}
Oct 30 19:18:22.745820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.745535    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx" event={"ID":"a4687e07-e7bd-4a45-a89e-b7550ec67f95","Type":"ContainerDied","Data":"a88326849351e931c1a5058bcc22bfef357428cf078f5b5928eb923597f3763a"}
Oct 30 19:18:22.747329 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.747310    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:18:22.750782 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.750703    5704 generic.go:356] "Generic (PLEG): container finished" podID="f78f6217-90d4-405c-bed9-36b9e9911b7c" containerID="019fdd8c5e1d451d6489b46540b16af971713bb0002d517724577dabcc3b5622" exitCode=0
Oct 30 19:18:22.750782 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.750734    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" event={"ID":"f78f6217-90d4-405c-bed9-36b9e9911b7c","Type":"ContainerDied","Data":"019fdd8c5e1d451d6489b46540b16af971713bb0002d517724577dabcc3b5622"}
Oct 30 19:18:22.761650 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.761624    5704 scope.go:122] "RemoveContainer" containerID="da33beb1ece53ce055e4b336ea7dc797298a21eec23816da07ace3f83af73a30"
Oct 30 19:18:22.778228 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.778022    5704 scope.go:122] "RemoveContainer" containerID="da33beb1ece53ce055e4b336ea7dc797298a21eec23816da07ace3f83af73a30"
Oct 30 19:18:22.778803 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:22.778759    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da33beb1ece53ce055e4b336ea7dc797298a21eec23816da07ace3f83af73a30\": container with ID starting with da33beb1ece53ce055e4b336ea7dc797298a21eec23816da07ace3f83af73a30 not found: ID does not exist" containerID="da33beb1ece53ce055e4b336ea7dc797298a21eec23816da07ace3f83af73a30"
Oct 30 19:18:22.779003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.778972    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da33beb1ece53ce055e4b336ea7dc797298a21eec23816da07ace3f83af73a30"} err="failed to get container status \"da33beb1ece53ce055e4b336ea7dc797298a21eec23816da07ace3f83af73a30\": rpc error: code = NotFound desc = could not find container \"da33beb1ece53ce055e4b336ea7dc797298a21eec23816da07ace3f83af73a30\": container with ID starting with da33beb1ece53ce055e4b336ea7dc797298a21eec23816da07ace3f83af73a30 not found: ID does not exist"
Oct 30 19:18:22.797850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.797826    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istiod-ca-cert\") pod \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") "
Oct 30 19:18:22.797970 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.797913    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-data\") pod \"28fe6291-de37-42c5-abe2-76e43b0f89b3\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") "
Oct 30 19:18:22.797970 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.797961    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-socket\") pod \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") "
Oct 30 19:18:22.798098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798051    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-kube-api-access-fpp68\") pod \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") "
Oct 30 19:18:22.798160 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798113    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-token\" (UniqueName: \"kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-token\") pod \"28fe6291-de37-42c5-abe2-76e43b0f89b3\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") "
Oct 30 19:18:22.798217 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798165    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-envoy\") pod \"28fe6291-de37-42c5-abe2-76e43b0f89b3\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") "
Oct 30 19:18:22.798217 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798191    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-data" pod "28fe6291-de37-42c5-abe2-76e43b0f89b3" (UID: "28fe6291-de37-42c5-abe2-76e43b0f89b3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.798323 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798203    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-kube-api-access-6wr87\" (UniqueName: \"kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-kube-api-access-6wr87\") pod \"28fe6291-de37-42c5-abe2-76e43b0f89b3\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") "
Oct 30 19:18:22.798323 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798292    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istiod-ca-cert" pod "eb6977e8-1680-449b-ad3e-3434eac1e0e9" (UID: "eb6977e8-1680-449b-ad3e-3434eac1e0e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:22.798323 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798302    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-certs\") pod \"28fe6291-de37-42c5-abe2-76e43b0f89b3\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") "
Oct 30 19:18:22.798474 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798319    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-socket" pod "eb6977e8-1680-449b-ad3e-3434eac1e0e9" (UID: "eb6977e8-1680-449b-ad3e-3434eac1e0e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.798474 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798332    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-certs\") pod \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") "
Oct 30 19:18:22.798474 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798412    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-envoy\") pod \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") "
Oct 30 19:18:22.798631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798471    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-credential-socket\") pod \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") "
Oct 30 19:18:22.798631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798492    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-certs" pod "eb6977e8-1680-449b-ad3e-3434eac1e0e9" (UID: "eb6977e8-1680-449b-ad3e-3434eac1e0e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.798631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798553    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-token\" (UniqueName: \"kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-token\") pod \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") "
Oct 30 19:18:22.798631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798596    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/28fe6291-de37-42c5-abe2-76e43b0f89b3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28fe6291-de37-42c5-abe2-76e43b0f89b3-istiod-ca-cert\") pod \"28fe6291-de37-42c5-abe2-76e43b0f89b3\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") "
Oct 30 19:18:22.798631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798626    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-socket\") pod \"28fe6291-de37-42c5-abe2-76e43b0f89b3\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") "
Oct 30 19:18:22.798869 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798632    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-certs" pod "28fe6291-de37-42c5-abe2-76e43b0f89b3" (UID: "28fe6291-de37-42c5-abe2-76e43b0f89b3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.798869 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798667    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-data\") pod \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") "
Oct 30 19:18:22.798869 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798703    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-credential-socket\") pod \"28fe6291-de37-42c5-abe2-76e43b0f89b3\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") "
Oct 30 19:18:22.798869 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798733    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-podinfo\") pod \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\" (UID: \"eb6977e8-1680-449b-ad3e-3434eac1e0e9\") "
Oct 30 19:18:22.798869 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.798802    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-podinfo\") pod \"28fe6291-de37-42c5-abe2-76e43b0f89b3\" (UID: \"28fe6291-de37-42c5-abe2-76e43b0f89b3\") "
Oct 30 19:18:22.799532 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.799510    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.799632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.799537    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.799632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.799551    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.799632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.799564    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.799632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.799578    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.800393 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.800364    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-credential-socket" pod "eb6977e8-1680-449b-ad3e-3434eac1e0e9" (UID: "eb6977e8-1680-449b-ad3e-3434eac1e0e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.800768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.800741    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28fe6291-de37-42c5-abe2-76e43b0f89b3-istiod-ca-cert" pod "28fe6291-de37-42c5-abe2-76e43b0f89b3" (UID: "28fe6291-de37-42c5-abe2-76e43b0f89b3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:22.801409 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.801387    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-socket" pod "28fe6291-de37-42c5-abe2-76e43b0f89b3" (UID: "28fe6291-de37-42c5-abe2-76e43b0f89b3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.801500 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.801459    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-data" pod "eb6977e8-1680-449b-ad3e-3434eac1e0e9" (UID: "eb6977e8-1680-449b-ad3e-3434eac1e0e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.801500 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.801474    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-credential-socket" pod "28fe6291-de37-42c5-abe2-76e43b0f89b3" (UID: "28fe6291-de37-42c5-abe2-76e43b0f89b3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.803136 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.801515    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-token" pod "28fe6291-de37-42c5-abe2-76e43b0f89b3" (UID: "28fe6291-de37-42c5-abe2-76e43b0f89b3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:22.803136 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.801551    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-kube-api-access-fpp68" pod "eb6977e8-1680-449b-ad3e-3434eac1e0e9" (UID: "eb6977e8-1680-449b-ad3e-3434eac1e0e9"). InnerVolumeSpecName "kube-api-access-fpp68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:22.803136 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.801574    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-envoy" pod "28fe6291-de37-42c5-abe2-76e43b0f89b3" (UID: "28fe6291-de37-42c5-abe2-76e43b0f89b3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.803136 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.802951    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:22.803862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.803833    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-podinfo" pod "eb6977e8-1680-449b-ad3e-3434eac1e0e9" (UID: "eb6977e8-1680-449b-ad3e-3434eac1e0e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:18:22.804420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.804390    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-envoy" pod "eb6977e8-1680-449b-ad3e-3434eac1e0e9" (UID: "eb6977e8-1680-449b-ad3e-3434eac1e0e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.804655 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.804629    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-kube-api-access-6wr87" pod "28fe6291-de37-42c5-abe2-76e43b0f89b3" (UID: "28fe6291-de37-42c5-abe2-76e43b0f89b3"). InnerVolumeSpecName "kube-api-access-6wr87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:22.805470 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.805445    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-token" pod "eb6977e8-1680-449b-ad3e-3434eac1e0e9" (UID: "eb6977e8-1680-449b-ad3e-3434eac1e0e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:22.805872 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.805851    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-podinfo" pod "28fe6291-de37-42c5-abe2-76e43b0f89b3" (UID: "28fe6291-de37-42c5-abe2-76e43b0f89b3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:18:22.820880 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.820857    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:18:22.822965 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.822949    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:18:22.900897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.900863    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-socket\") pod \"307212d3-c2b4-4bc6-908e-607728b8e195\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") "
Oct 30 19:18:22.901132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.900950    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-podinfo\") pod \"f78f6217-90d4-405c-bed9-36b9e9911b7c\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") "
Oct 30 19:18:22.901132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901001    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-kube-api-access-thgln\" (UniqueName: \"kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-kube-api-access-thgln\") pod \"307212d3-c2b4-4bc6-908e-607728b8e195\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") "
Oct 30 19:18:22.901132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901048    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-token\" (UniqueName: \"kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-token\") pod \"f78f6217-90d4-405c-bed9-36b9e9911b7c\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") "
Oct 30 19:18:22.901132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901119    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/307212d3-c2b4-4bc6-908e-607728b8e195-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/307212d3-c2b4-4bc6-908e-607728b8e195-istiod-ca-cert\") pod \"307212d3-c2b4-4bc6-908e-607728b8e195\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") "
Oct 30 19:18:22.901394 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901152    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-envoy\") pod \"f78f6217-90d4-405c-bed9-36b9e9911b7c\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") "
Oct 30 19:18:22.901394 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901217    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-socket" pod "307212d3-c2b4-4bc6-908e-607728b8e195" (UID: "307212d3-c2b4-4bc6-908e-607728b8e195"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901648    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-credential-socket\") pod \"307212d3-c2b4-4bc6-908e-607728b8e195\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") "
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901699    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/307212d3-c2b4-4bc6-908e-607728b8e195-istiod-ca-cert" pod "307212d3-c2b4-4bc6-908e-607728b8e195" (UID: "307212d3-c2b4-4bc6-908e-607728b8e195"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901728    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-envoy\") pod \"307212d3-c2b4-4bc6-908e-607728b8e195\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") "
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901759    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-certs\") pod \"307212d3-c2b4-4bc6-908e-607728b8e195\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") "
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901823    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-socket\") pod \"f78f6217-90d4-405c-bed9-36b9e9911b7c\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") "
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901901    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/307212d3-c2b4-4bc6-908e-607728b8e195-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/307212d3-c2b4-4bc6-908e-607728b8e195-istio-podinfo\") pod \"307212d3-c2b4-4bc6-908e-607728b8e195\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") "
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901932    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-certs\") pod \"f78f6217-90d4-405c-bed9-36b9e9911b7c\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") "
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901949    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-credential-socket" pod "307212d3-c2b4-4bc6-908e-607728b8e195" (UID: "307212d3-c2b4-4bc6-908e-607728b8e195"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.901968    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-data\") pod \"307212d3-c2b4-4bc6-908e-607728b8e195\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") "
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902001    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-kube-api-access-cl28x\" (UniqueName: \"kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-kube-api-access-cl28x\") pod \"f78f6217-90d4-405c-bed9-36b9e9911b7c\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") "
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902064    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-data\") pod \"f78f6217-90d4-405c-bed9-36b9e9911b7c\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") "
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902107    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f78f6217-90d4-405c-bed9-36b9e9911b7c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f78f6217-90d4-405c-bed9-36b9e9911b7c-istiod-ca-cert\") pod \"f78f6217-90d4-405c-bed9-36b9e9911b7c\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") "
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902142    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-credential-socket\") pod \"f78f6217-90d4-405c-bed9-36b9e9911b7c\" (UID: \"f78f6217-90d4-405c-bed9-36b9e9911b7c\") "
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902159    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-socket" pod "f78f6217-90d4-405c-bed9-36b9e9911b7c" (UID: "f78f6217-90d4-405c-bed9-36b9e9911b7c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902180    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-istio-token\" (UniqueName: \"kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-istio-token\") pod \"307212d3-c2b4-4bc6-908e-607728b8e195\" (UID: \"307212d3-c2b4-4bc6-908e-607728b8e195\") "
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902622    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-certs" pod "307212d3-c2b4-4bc6-908e-607728b8e195" (UID: "307212d3-c2b4-4bc6-908e-607728b8e195"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902853    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/307212d3-c2b4-4bc6-908e-607728b8e195-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902873    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902891    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902908    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902923    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902938    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/28fe6291-de37-42c5-abe2-76e43b0f89b3-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902957    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902971    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.902986    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.903000    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.903013    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb6977e8-1680-449b-ad3e-3434eac1e0e9-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.903028    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.903077    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.903093    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.903109    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpp68\" (UniqueName: \"kubernetes.io/projected/eb6977e8-1680-449b-ad3e-3434eac1e0e9-kube-api-access-fpp68\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.903124    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.903140    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/28fe6291-de37-42c5-abe2-76e43b0f89b3-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.903149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.903155    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wr87\" (UniqueName: \"kubernetes.io/projected/28fe6291-de37-42c5-abe2-76e43b0f89b3-kube-api-access-6wr87\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:22.905152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.903406    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-certs" pod "f78f6217-90d4-405c-bed9-36b9e9911b7c" (UID: "f78f6217-90d4-405c-bed9-36b9e9911b7c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.905152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.903766    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-podinfo" pod "f78f6217-90d4-405c-bed9-36b9e9911b7c" (UID: "f78f6217-90d4-405c-bed9-36b9e9911b7c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:18:22.905152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.904146    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-data" pod "307212d3-c2b4-4bc6-908e-607728b8e195" (UID: "307212d3-c2b4-4bc6-908e-607728b8e195"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.905152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.904152    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-data" pod "f78f6217-90d4-405c-bed9-36b9e9911b7c" (UID: "f78f6217-90d4-405c-bed9-36b9e9911b7c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.905152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.904167    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f78f6217-90d4-405c-bed9-36b9e9911b7c-istiod-ca-cert" pod "f78f6217-90d4-405c-bed9-36b9e9911b7c" (UID: "f78f6217-90d4-405c-bed9-36b9e9911b7c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:22.906128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.905754    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-credential-socket" pod "f78f6217-90d4-405c-bed9-36b9e9911b7c" (UID: "f78f6217-90d4-405c-bed9-36b9e9911b7c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.906128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.905822    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-istio-token" pod "307212d3-c2b4-4bc6-908e-607728b8e195" (UID: "307212d3-c2b4-4bc6-908e-607728b8e195"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:22.906128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.905925    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-token" pod "f78f6217-90d4-405c-bed9-36b9e9911b7c" (UID: "f78f6217-90d4-405c-bed9-36b9e9911b7c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:22.906128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.905957    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-envoy" pod "307212d3-c2b4-4bc6-908e-607728b8e195" (UID: "307212d3-c2b4-4bc6-908e-607728b8e195"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.906128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.905960    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-kube-api-access-thgln" pod "307212d3-c2b4-4bc6-908e-607728b8e195" (UID: "307212d3-c2b4-4bc6-908e-607728b8e195"). InnerVolumeSpecName "kube-api-access-thgln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:22.906128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.905976    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-envoy" pod "f78f6217-90d4-405c-bed9-36b9e9911b7c" (UID: "f78f6217-90d4-405c-bed9-36b9e9911b7c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:18:22.906595 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.906566    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/307212d3-c2b4-4bc6-908e-607728b8e195-istio-podinfo" pod "307212d3-c2b4-4bc6-908e-607728b8e195" (UID: "307212d3-c2b4-4bc6-908e-607728b8e195"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:18:22.918929 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:22.918897    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-kube-api-access-cl28x" pod "f78f6217-90d4-405c-bed9-36b9e9911b7c" (UID: "f78f6217-90d4-405c-bed9-36b9e9911b7c"). InnerVolumeSpecName "kube-api-access-cl28x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:23.004327 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.004294    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.004475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.004332    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.004475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.004348    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/307212d3-c2b4-4bc6-908e-607728b8e195-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.004475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.004363    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.004475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.004377    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/307212d3-c2b4-4bc6-908e-607728b8e195-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.004475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.004391    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cl28x\" (UniqueName: \"kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-kube-api-access-cl28x\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.004475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.004405    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.004475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.004419    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f78f6217-90d4-405c-bed9-36b9e9911b7c-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.004475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.004433    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f78f6217-90d4-405c-bed9-36b9e9911b7c-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.004475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.004446    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.004475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.004458    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.004475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.004471    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thgln\" (UniqueName: \"kubernetes.io/projected/307212d3-c2b4-4bc6-908e-607728b8e195-kube-api-access-thgln\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.004811 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.004485    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f78f6217-90d4-405c-bed9-36b9e9911b7c-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:23.058260 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.058223    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:23.237758 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.237730    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"]
Oct 30 19:18:23.242949 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.242919    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-z8tnx"]
Oct 30 19:18:23.762582 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.762547    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"
Oct 30 19:18:23.763028 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.762546    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq" event={"ID":"eb6977e8-1680-449b-ad3e-3434eac1e0e9","Type":"ContainerDied","Data":"7716e6ee6624575d1f78a5ee019d82efe0e5305f281d691f19b75343121ce48c"}
Oct 30 19:18:23.763028 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.762677    5704 scope.go:122] "RemoveContainer" containerID="82f89b20be049ce7a1db619a6dc4f0281895300c03aa97766acaa4bc34c62ab9"
Oct 30 19:18:23.764085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.763987    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"
Oct 30 19:18:23.764085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.764017    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q" event={"ID":"28fe6291-de37-42c5-abe2-76e43b0f89b3","Type":"ContainerDied","Data":"155d8c8b897fc675caa7747b154a1d68bd1aa42ff96377f0b29e9e2d23408cb9"}
Oct 30 19:18:23.765738 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.765720    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"
Oct 30 19:18:23.765859 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.765733    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562" event={"ID":"307212d3-c2b4-4bc6-908e-607728b8e195","Type":"ContainerDied","Data":"83d137b8c083740759fd6c1346f990d1655239225b91978b1b6473a9b3aeaf72"}
Oct 30 19:18:23.768532 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.768510    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t" event={"ID":"f78f6217-90d4-405c-bed9-36b9e9911b7c","Type":"ContainerDied","Data":"5f28919f1d85606ef12d876df30addf517031007c943caa3b5d224cd28f46e37"}
Oct 30 19:18:23.768619 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.768561    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"
Oct 30 19:18:23.774015 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.773998    5704 scope.go:122] "RemoveContainer" containerID="c66ace058f3d7733b1569364c72118482848894dafdce04193369e979c66fbab"
Oct 30 19:18:23.782762 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.782731    5704 scope.go:122] "RemoveContainer" containerID="019fdd8c5e1d451d6489b46540b16af971713bb0002d517724577dabcc3b5622"
Oct 30 19:18:23.917584 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.917560    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"]
Oct 30 19:18:23.922260 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:23.922239    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l2cgq"]
Oct 30 19:18:24.025402 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:24.025323    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"]
Oct 30 19:18:24.030129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:24.030099    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-gp562"]
Oct 30 19:18:24.133256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:24.133234    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"]
Oct 30 19:18:24.136946 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:24.136924    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-nsh4q"]
Oct 30 19:18:24.243435 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:24.243404    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"]
Oct 30 19:18:24.247952 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:24.247929    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-k7q6t"]
Oct 30 19:18:25.180593 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:25.180560    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="28fe6291-de37-42c5-abe2-76e43b0f89b3" path="/var/lib/kubelet/pods/28fe6291-de37-42c5-abe2-76e43b0f89b3/volumes"
Oct 30 19:18:25.181024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:25.181009    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="307212d3-c2b4-4bc6-908e-607728b8e195" path="/var/lib/kubelet/pods/307212d3-c2b4-4bc6-908e-607728b8e195/volumes"
Oct 30 19:18:25.181456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:25.181443    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a4687e07-e7bd-4a45-a89e-b7550ec67f95" path="/var/lib/kubelet/pods/a4687e07-e7bd-4a45-a89e-b7550ec67f95/volumes"
Oct 30 19:18:25.182192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:25.182176    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" path="/var/lib/kubelet/pods/eb6977e8-1680-449b-ad3e-3434eac1e0e9/volumes"
Oct 30 19:18:25.182587 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:25.182576    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f78f6217-90d4-405c-bed9-36b9e9911b7c" path="/var/lib/kubelet/pods/f78f6217-90d4-405c-bed9-36b9e9911b7c/volumes"
Oct 30 19:18:27.981613 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:27.981580    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:18:28.236659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:28.236588    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:33.384461 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:33.384417    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:18:33.441722 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:33.441626    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:37.426907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.426871    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"]
Oct 30 19:18:37.427367 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.427079    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k" podUID="c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884" containerName="maas-api" containerID="cri-o://eb788f66ccbb4b911f4ac4d1f9ea60821997a6c131bdb598d3aeaa3f8506cbbc" gracePeriod=30
Oct 30 19:18:37.564108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.564075    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:37.739972 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.739945    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:18:37.885950 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.885899    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-maas-api-tls\") pod \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") "
Oct 30 19:18:37.886192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.885974    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-metrics-certs\") pod \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") "
Oct 30 19:18:37.886192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.886016    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-kube-api-access-k9pnk\" (UniqueName: \"kubernetes.io/projected/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-kube-api-access-k9pnk\") pod \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") "
Oct 30 19:18:37.886192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.886145    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-openshift-service-ca\") pod \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\" (UID: \"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884\") "
Oct 30 19:18:37.887614 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.887583    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-openshift-service-ca" pod "c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884" (UID: "c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:18:37.889170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.889146    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-metrics-certs" pod "c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884" (UID: "c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:18:37.889548 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.889527    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-kube-api-access-k9pnk" pod "c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884" (UID: "c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884"). InnerVolumeSpecName "kube-api-access-k9pnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:37.895829 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.895449    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-maas-api-tls" pod "c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884" (UID: "c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:18:37.903896 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.903848    5704 generic.go:356] "Generic (PLEG): container finished" podID="c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884" containerID="eb788f66ccbb4b911f4ac4d1f9ea60821997a6c131bdb598d3aeaa3f8506cbbc" exitCode=0
Oct 30 19:18:37.904023 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.903930    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"
Oct 30 19:18:37.904023 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.903945    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k" event={"ID":"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884","Type":"ContainerDied","Data":"eb788f66ccbb4b911f4ac4d1f9ea60821997a6c131bdb598d3aeaa3f8506cbbc"}
Oct 30 19:18:37.904023 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.903999    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k" event={"ID":"c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884","Type":"ContainerDied","Data":"91ae6a7d3fb5903289fcc0963fffebb402a421663605b5a3909d5e9eb1aaae3d"}
Oct 30 19:18:37.904023 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.904044    5704 scope.go:122] "RemoveContainer" containerID="eb788f66ccbb4b911f4ac4d1f9ea60821997a6c131bdb598d3aeaa3f8506cbbc"
Oct 30 19:18:37.919573 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.919546    5704 scope.go:122] "RemoveContainer" containerID="eb788f66ccbb4b911f4ac4d1f9ea60821997a6c131bdb598d3aeaa3f8506cbbc"
Oct 30 19:18:37.919866 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:37.919847    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eb788f66ccbb4b911f4ac4d1f9ea60821997a6c131bdb598d3aeaa3f8506cbbc\": container with ID starting with eb788f66ccbb4b911f4ac4d1f9ea60821997a6c131bdb598d3aeaa3f8506cbbc not found: ID does not exist" containerID="eb788f66ccbb4b911f4ac4d1f9ea60821997a6c131bdb598d3aeaa3f8506cbbc"
Oct 30 19:18:37.919917 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:37.919876    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eb788f66ccbb4b911f4ac4d1f9ea60821997a6c131bdb598d3aeaa3f8506cbbc"} err="failed to get container status \"eb788f66ccbb4b911f4ac4d1f9ea60821997a6c131bdb598d3aeaa3f8506cbbc\": rpc error: code = NotFound desc = could not find container \"eb788f66ccbb4b911f4ac4d1f9ea60821997a6c131bdb598d3aeaa3f8506cbbc\": container with ID starting with eb788f66ccbb4b911f4ac4d1f9ea60821997a6c131bdb598d3aeaa3f8506cbbc not found: ID does not exist"
Oct 30 19:18:38.001109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.000743    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k9pnk\" (UniqueName: \"kubernetes.io/projected/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-kube-api-access-k9pnk\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:38.001109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.000774    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:38.001109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.000789    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:38.001109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.000803    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:38.073395 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.073360    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"]
Oct 30 19:18:38.077281 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.077257    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0d2ef6-66d6bb9bc6-4vv6k"]
Oct 30 19:18:38.555726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.555689    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7402-aaf460"]
Oct 30 19:18:38.556285 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556270    5704 cpu_manager.go:414] "containerMap: removing container" podUID="307212d3-c2b4-4bc6-908e-607728b8e195" containerName="istio-proxy"
Oct 30 19:18:38.556334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556287    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="307212d3-c2b4-4bc6-908e-607728b8e195" containerName="istio-proxy"
Oct 30 19:18:38.556334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556306    5704 cpu_manager.go:414] "containerMap: removing container" podUID="fe2b1ef6-71e9-470a-ae30-aa957f4c84b6" containerName="payload-pre-processing"
Oct 30 19:18:38.556334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556312    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fe2b1ef6-71e9-470a-ae30-aa957f4c84b6" containerName="payload-pre-processing"
Oct 30 19:18:38.556334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556322    5704 cpu_manager.go:414] "containerMap: removing container" podUID="1324e64f-d43b-46dc-bcbf-7d5d4565b3df" containerName="maas-api"
Oct 30 19:18:38.556334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556327    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1324e64f-d43b-46dc-bcbf-7d5d4565b3df" containerName="maas-api"
Oct 30 19:18:38.556334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556334    5704 cpu_manager.go:414] "containerMap: removing container" podUID="c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884" containerName="maas-api"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556338    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884" containerName="maas-api"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556350    5704 cpu_manager.go:414] "containerMap: removing container" podUID="f78f6217-90d4-405c-bed9-36b9e9911b7c" containerName="istio-proxy"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556355    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f78f6217-90d4-405c-bed9-36b9e9911b7c" containerName="istio-proxy"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556363    5704 cpu_manager.go:414] "containerMap: removing container" podUID="2a762f75-0170-4a91-bcf4-bb2af0079517" containerName="payload-processing"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556367    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a762f75-0170-4a91-bcf4-bb2af0079517" containerName="payload-processing"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556377    5704 cpu_manager.go:414] "containerMap: removing container" podUID="a4687e07-e7bd-4a45-a89e-b7550ec67f95" containerName="istio-proxy"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556381    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a4687e07-e7bd-4a45-a89e-b7550ec67f95" containerName="istio-proxy"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556391    5704 cpu_manager.go:414] "containerMap: removing container" podUID="be13b502-20cc-4754-ab38-71eee24dca18" containerName="payload-pre-processing"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556396    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="be13b502-20cc-4754-ab38-71eee24dca18" containerName="payload-pre-processing"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556418    5704 cpu_manager.go:414] "containerMap: removing container" podUID="28fe6291-de37-42c5-abe2-76e43b0f89b3" containerName="istio-proxy"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556422    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="28fe6291-de37-42c5-abe2-76e43b0f89b3" containerName="istio-proxy"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556431    5704 cpu_manager.go:414] "containerMap: removing container" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerName="istio-proxy"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556435    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerName="istio-proxy"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556441    5704 cpu_manager.go:414] "containerMap: removing container" podUID="42b62545-91fd-4215-9c57-b1273c5471d6" containerName="istio-proxy"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556445    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="42b62545-91fd-4215-9c57-b1273c5471d6" containerName="istio-proxy"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556451    5704 cpu_manager.go:414] "containerMap: removing container" podUID="34886f9b-18a4-4889-bef9-5b1c16f8ce2e" containerName="istio-proxy"
Oct 30 19:18:38.556512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556456    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="34886f9b-18a4-4889-bef9-5b1c16f8ce2e" containerName="istio-proxy"
Oct 30 19:18:38.557003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556529    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="307212d3-c2b4-4bc6-908e-607728b8e195" containerName="istio-proxy"
Oct 30 19:18:38.557003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556540    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="fe2b1ef6-71e9-470a-ae30-aa957f4c84b6" containerName="payload-pre-processing"
Oct 30 19:18:38.557003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556547    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="1324e64f-d43b-46dc-bcbf-7d5d4565b3df" containerName="maas-api"
Oct 30 19:18:38.557003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556552    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884" containerName="maas-api"
Oct 30 19:18:38.557003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556559    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="f78f6217-90d4-405c-bed9-36b9e9911b7c" containerName="istio-proxy"
Oct 30 19:18:38.557003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556566    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a762f75-0170-4a91-bcf4-bb2af0079517" containerName="payload-processing"
Oct 30 19:18:38.557003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556572    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="a4687e07-e7bd-4a45-a89e-b7550ec67f95" containerName="istio-proxy"
Oct 30 19:18:38.557003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556579    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="be13b502-20cc-4754-ab38-71eee24dca18" containerName="payload-pre-processing"
Oct 30 19:18:38.557003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556588    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="28fe6291-de37-42c5-abe2-76e43b0f89b3" containerName="istio-proxy"
Oct 30 19:18:38.557003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556593    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb6977e8-1680-449b-ad3e-3434eac1e0e9" containerName="istio-proxy"
Oct 30 19:18:38.557003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556599    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="42b62545-91fd-4215-9c57-b1273c5471d6" containerName="istio-proxy"
Oct 30 19:18:38.557003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.556605    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="34886f9b-18a4-4889-bef9-5b1c16f8ce2e" containerName="istio-proxy"
Oct 30 19:18:38.561587 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.561565    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:38.610546 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.610522    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7hnn9\""
Oct 30 19:18:38.616100 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.616082    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7402-aaf460"]
Oct 30 19:18:38.709525 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.709500    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5lpc\" (UniqueName: \"kubernetes.io/projected/f5a0a6ec-2d19-4348-88f1-7370d80db8cf-kube-api-access-b5lpc\") pod \"test-curl-7402-aaf460\" (UID: \"f5a0a6ec-2d19-4348-88f1-7370d80db8cf\") " pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:38.810422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.810359    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5lpc\" (UniqueName: \"kubernetes.io/projected/f5a0a6ec-2d19-4348-88f1-7370d80db8cf-kube-api-access-b5lpc\") pod \"test-curl-7402-aaf460\" (UID: \"f5a0a6ec-2d19-4348-88f1-7370d80db8cf\") " pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:38.867731 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.867704    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5lpc\" (UniqueName: \"kubernetes.io/projected/f5a0a6ec-2d19-4348-88f1-7370d80db8cf-kube-api-access-b5lpc\") pod \"test-curl-7402-aaf460\" (UID: \"f5a0a6ec-2d19-4348-88f1-7370d80db8cf\") " pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:38.873638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:38.873608    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:39.008610 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:39.008581    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:39.179182 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:39.179111    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884" path="/var/lib/kubelet/pods/c10a8ae1-4bbe-4b2e-817a-01ebfd1fa884/volumes"
Oct 30 19:18:39.235229 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:39.235194    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:39.245805 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:39.245787    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7402-aaf460"]
Oct 30 19:18:39.248099 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:18:39.248072    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf5a0a6ec_2d19_4348_88f1_7370d80db8cf.slice/crio-9309bc05da7e5b27d787dfaf7bdfcdcc796deee9da6a361c29ef3fa5ea7ac6e1 WatchSource:0}: Error finding container 9309bc05da7e5b27d787dfaf7bdfcdcc796deee9da6a361c29ef3fa5ea7ac6e1: Status 404 returned error can't find the container with id 9309bc05da7e5b27d787dfaf7bdfcdcc796deee9da6a361c29ef3fa5ea7ac6e1
Oct 30 19:18:39.249867 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:39.249849    5704 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:18:39.415432 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:39.415396    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:39.664058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:39.663469    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:39.911331 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:39.911294    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:39.924147 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:39.924078    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7402-aaf460" event={"ID":"f5a0a6ec-2d19-4348-88f1-7370d80db8cf","Type":"ContainerStarted","Data":"9309bc05da7e5b27d787dfaf7bdfcdcc796deee9da6a361c29ef3fa5ea7ac6e1"}
Oct 30 19:18:40.103185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:40.103150    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:42.947299 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:42.947260    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7402-aaf460" event={"ID":"f5a0a6ec-2d19-4348-88f1-7370d80db8cf","Type":"ContainerStarted","Data":"893fe148ba125072802fd579f954d4c6cb991140363ce8fc5dd4ed6f4a314020"}
Oct 30 19:18:43.055528 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:43.055481    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-7402-aaf460" podStartSLOduration=2.168148748 podStartE2EDuration="5.055468217s" totalImagesPullingTime="2.887319469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:43.053444327 +0000 UTC m=+1895.999612229" watchObservedRunningTime="2026-10-30 19:18:43.055468217 +0000 UTC m=+1896.001636119"
Oct 30 19:18:43.673881 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:43.673838    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:43.876151 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:43.876114    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:44.057198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:44.057163    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:44.325813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:44.325723    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:44.625504 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:44.625425    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7402-aaf460"]
Oct 30 19:18:44.627782 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:44.627760    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7402-aaf460"]
Oct 30 19:18:44.962706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:44.962614    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-7402-aaf460" podUID="f5a0a6ec-2d19-4348-88f1-7370d80db8cf" containerName="test-curl-7402-aaf460" containerID="cri-o://893fe148ba125072802fd579f954d4c6cb991140363ce8fc5dd4ed6f4a314020" gracePeriod=2
Oct 30 19:18:45.010527 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:45.010489    5704 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7402-aaf460\" is forbidden: User \"system:node:ip-10-0-139-33.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-33.us-west-2.compute.internal' and this object" podUID="f5a0a6ec-2d19-4348-88f1-7370d80db8cf" pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:46.929056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:46.923117    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:47.136133 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:47.136092    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:47.234222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:47.234193    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:47.281942 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:47.281913    5704 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7402-aaf460\" is forbidden: User \"system:node:ip-10-0-139-33.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-33.us-west-2.compute.internal' and this object" podUID="f5a0a6ec-2d19-4348-88f1-7370d80db8cf" pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:47.349107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:47.349083    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f5a0a6ec-2d19-4348-88f1-7370d80db8cf-kube-api-access-b5lpc\" (UniqueName: \"kubernetes.io/projected/f5a0a6ec-2d19-4348-88f1-7370d80db8cf-kube-api-access-b5lpc\") pod \"f5a0a6ec-2d19-4348-88f1-7370d80db8cf\" (UID: \"f5a0a6ec-2d19-4348-88f1-7370d80db8cf\") "
Oct 30 19:18:47.392243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:47.392202    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5a0a6ec-2d19-4348-88f1-7370d80db8cf-kube-api-access-b5lpc" pod "f5a0a6ec-2d19-4348-88f1-7370d80db8cf" (UID: "f5a0a6ec-2d19-4348-88f1-7370d80db8cf"). InnerVolumeSpecName "kube-api-access-b5lpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:18:47.420906 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:47.420873    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:47.455045 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:47.454992    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5lpc\" (UniqueName: \"kubernetes.io/projected/f5a0a6ec-2d19-4348-88f1-7370d80db8cf-kube-api-access-b5lpc\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:18:47.638440 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:47.638400    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:47.821663 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:47.821629    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:47.983355 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:47.983241    5704 generic.go:356] "Generic (PLEG): container finished" podID="f5a0a6ec-2d19-4348-88f1-7370d80db8cf" containerID="893fe148ba125072802fd579f954d4c6cb991140363ce8fc5dd4ed6f4a314020" exitCode=137
Oct 30 19:18:47.983355 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:47.983285    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:47.983355 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:47.983346    5704 scope.go:122] "RemoveContainer" containerID="893fe148ba125072802fd579f954d4c6cb991140363ce8fc5dd4ed6f4a314020"
Oct 30 19:18:47.993138 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:47.993115    5704 scope.go:122] "RemoveContainer" containerID="893fe148ba125072802fd579f954d4c6cb991140363ce8fc5dd4ed6f4a314020"
Oct 30 19:18:47.993405 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:47.993383    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"893fe148ba125072802fd579f954d4c6cb991140363ce8fc5dd4ed6f4a314020\": container with ID starting with 893fe148ba125072802fd579f954d4c6cb991140363ce8fc5dd4ed6f4a314020 not found: ID does not exist" containerID="893fe148ba125072802fd579f954d4c6cb991140363ce8fc5dd4ed6f4a314020"
Oct 30 19:18:47.993475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:47.993413    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"893fe148ba125072802fd579f954d4c6cb991140363ce8fc5dd4ed6f4a314020"} err="failed to get container status \"893fe148ba125072802fd579f954d4c6cb991140363ce8fc5dd4ed6f4a314020\": rpc error: code = NotFound desc = could not find container \"893fe148ba125072802fd579f954d4c6cb991140363ce8fc5dd4ed6f4a314020\": container with ID starting with 893fe148ba125072802fd579f954d4c6cb991140363ce8fc5dd4ed6f4a314020 not found: ID does not exist"
Oct 30 19:18:48.025238 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.025204    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:48.030697 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:48.030673    5704 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7402-aaf460\" is forbidden: User \"system:node:ip-10-0-139-33.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-33.us-west-2.compute.internal' and this object" podUID="f5a0a6ec-2d19-4348-88f1-7370d80db8cf" pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:48.085055 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:48.080333    5704 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7402-aaf460\" is forbidden: User \"system:node:ip-10-0-139-33.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-33.us-west-2.compute.internal' and this object" podUID="f5a0a6ec-2d19-4348-88f1-7370d80db8cf" pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:48.160763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.160733    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"]
Oct 30 19:18:48.161665 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.161636    5704 cpu_manager.go:414] "containerMap: removing container" podUID="f5a0a6ec-2d19-4348-88f1-7370d80db8cf" containerName="test-curl-7402-aaf460"
Oct 30 19:18:48.161665 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.161663    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f5a0a6ec-2d19-4348-88f1-7370d80db8cf" containerName="test-curl-7402-aaf460"
Oct 30 19:18:48.161858 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.161803    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="f5a0a6ec-2d19-4348-88f1-7370d80db8cf" containerName="test-curl-7402-aaf460"
Oct 30 19:18:48.171518 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.171492    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.208464 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.208430    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"]
Oct 30 19:18:48.212375 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.212356    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"
Oct 30 19:18:48.217496 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.217471    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"]
Oct 30 19:18:48.219258 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:48.219228    5704 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7402-aaf460\" is forbidden: User \"system:node:ip-10-0-139-33.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-33.us-west-2.compute.internal' and this object" podUID="f5a0a6ec-2d19-4348-88f1-7370d80db8cf" pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:48.262252 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.262228    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-905566\""
Oct 30 19:18:48.266096 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.266079    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-4ec347-openshift-default-dockercfg-kpfgt\""
Oct 30 19:18:48.266156 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.266107    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-905566-dockercfg-r2hwp\""
Oct 30 19:18:48.266446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.266428    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crdw2\" (UniqueName: \"kubernetes.io/projected/eb1c92a8-55e3-46fd-ae18-410645416414-kube-api-access-crdw2\") pod \"payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb\" (UID: \"eb1c92a8-55e3-46fd-ae18-410645416414\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"
Oct 30 19:18:48.266481 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.266460    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1c92a8-55e3-46fd-ae18-410645416414-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb\" (UID: \"eb1c92a8-55e3-46fd-ae18-410645416414\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"
Oct 30 19:18:48.266523 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.266480    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-envoy\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.266523 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.266495    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-istio-token\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.266621 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.266596    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-credential-socket\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.266738 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.266636    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/793048dc-32db-41ad-9964-04c275d02cbf-istiod-ca-cert\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.266738 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.266669    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-certs\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.266815 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.266757    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kstrr\" (UniqueName: \"kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-kube-api-access-kstrr\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.266874 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.266860    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/793048dc-32db-41ad-9964-04c275d02cbf-istio-podinfo\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.266929 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.266918    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-socket\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.267005 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.266994    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-data\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.277911 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.277882    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"]
Oct 30 19:18:48.407335 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.407296    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kstrr\" (UniqueName: \"kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-kube-api-access-kstrr\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.407665 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.407644    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/793048dc-32db-41ad-9964-04c275d02cbf-istio-podinfo\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.407859 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.407839    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-socket\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.408082 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.408063    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-data\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.408208 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.408192    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crdw2\" (UniqueName: \"kubernetes.io/projected/eb1c92a8-55e3-46fd-ae18-410645416414-kube-api-access-crdw2\") pod \"payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb\" (UID: \"eb1c92a8-55e3-46fd-ae18-410645416414\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"
Oct 30 19:18:48.408316 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.408303    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1c92a8-55e3-46fd-ae18-410645416414-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb\" (UID: \"eb1c92a8-55e3-46fd-ae18-410645416414\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"
Oct 30 19:18:48.408410 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.408396    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-envoy\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.408502 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.408489    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-istio-token\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.408632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.408618    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-credential-socket\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.408771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.408752    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/793048dc-32db-41ad-9964-04c275d02cbf-istiod-ca-cert\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.408925 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.408909    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-certs\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.409090 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.408949    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-data\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.409199 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.408547    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-socket\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.409574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.409553    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-credential-socket\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.409700 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.409600    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1c92a8-55e3-46fd-ae18-410645416414-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb\" (UID: \"eb1c92a8-55e3-46fd-ae18-410645416414\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"
Oct 30 19:18:48.409798 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.407671    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:48.410601 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.410580    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/793048dc-32db-41ad-9964-04c275d02cbf-istiod-ca-cert\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.410986 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.410966    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-certs\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.419505 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.419425    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-envoy\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.420114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.420090    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/793048dc-32db-41ad-9964-04c275d02cbf-istio-podinfo\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.421002 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.420981    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"]
Oct 30 19:18:48.453238 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.451298    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"]
Oct 30 19:18:48.453238 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.451428    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"
Oct 30 19:18:48.459303 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:48.459263    5704 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7402-aaf460\" is forbidden: User \"system:node:ip-10-0-139-33.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-33.us-west-2.compute.internal' and this object" podUID="f5a0a6ec-2d19-4348-88f1-7370d80db8cf" pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:48.467726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.467243    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crdw2\" (UniqueName: \"kubernetes.io/projected/eb1c92a8-55e3-46fd-ae18-410645416414-kube-api-access-crdw2\") pod \"payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb\" (UID: \"eb1c92a8-55e3-46fd-ae18-410645416414\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"
Oct 30 19:18:48.468904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.468866    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-istio-token\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.469384 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.469366    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kstrr\" (UniqueName: \"kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-kube-api-access-kstrr\") pod \"e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") " pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.494248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.494221    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:48.509829 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.509802    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq5nb\" (UniqueName: \"kubernetes.io/projected/8db7e37a-9924-4ce9-bac0-67edd0374bf0-kube-api-access-mq5nb\") pod \"payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf\" (UID: \"8db7e37a-9924-4ce9-bac0-67edd0374bf0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"
Oct 30 19:18:48.509938 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.509895    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db7e37a-9924-4ce9-bac0-67edd0374bf0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf\" (UID: \"8db7e37a-9924-4ce9-bac0-67edd0374bf0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"
Oct 30 19:18:48.539943 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.539868    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"
Oct 30 19:18:48.594570 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.592109    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:48.610990 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.610939    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq5nb\" (UniqueName: \"kubernetes.io/projected/8db7e37a-9924-4ce9-bac0-67edd0374bf0-kube-api-access-mq5nb\") pod \"payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf\" (UID: \"8db7e37a-9924-4ce9-bac0-67edd0374bf0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"
Oct 30 19:18:48.611398 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.611100    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db7e37a-9924-4ce9-bac0-67edd0374bf0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf\" (UID: \"8db7e37a-9924-4ce9-bac0-67edd0374bf0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"
Oct 30 19:18:48.611945 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.611864    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db7e37a-9924-4ce9-bac0-67edd0374bf0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf\" (UID: \"8db7e37a-9924-4ce9-bac0-67edd0374bf0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"
Oct 30 19:18:48.614581 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:18:48.614233    5704 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-7402-aaf460\" is forbidden: User \"system:node:ip-10-0-139-33.us-west-2.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-33.us-west-2.compute.internal' and this object" podUID="f5a0a6ec-2d19-4348-88f1-7370d80db8cf" pod="opendatahub/test-curl-7402-aaf460"
Oct 30 19:18:48.668173 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.668150    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq5nb\" (UniqueName: \"kubernetes.io/projected/8db7e37a-9924-4ce9-bac0-67edd0374bf0-kube-api-access-mq5nb\") pod \"payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf\" (UID: \"8db7e37a-9924-4ce9-bac0-67edd0374bf0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"
Oct 30 19:18:48.820103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.817685    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"
Oct 30 19:18:48.848834 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.848790    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:48.925283 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.925256    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"]
Oct 30 19:18:48.928509 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:18:48.928195    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod793048dc_32db_41ad_9964_04c275d02cbf.slice/crio-cc041c28fbf0e1f767ccd4ba45f05847317cef72331a7711a34978e7e8ce5838 WatchSource:0}: Error finding container cc041c28fbf0e1f767ccd4ba45f05847317cef72331a7711a34978e7e8ce5838: Status 404 returned error can't find the container with id cc041c28fbf0e1f767ccd4ba45f05847317cef72331a7711a34978e7e8ce5838
Oct 30 19:18:48.930831 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.930797    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:18:48.930918 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.930863    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:18:48.930918 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.930890    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:18:48.972270 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.972228    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"]
Oct 30 19:18:48.983513 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:18:48.983464    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb1c92a8_55e3_46fd_ae18_410645416414.slice/crio-ac7b29796e636f0e2001dded5e1145c167edcc59adc4d77a05ee202a039d75c2 WatchSource:0}: Error finding container ac7b29796e636f0e2001dded5e1145c167edcc59adc4d77a05ee202a039d75c2: Status 404 returned error can't find the container with id ac7b29796e636f0e2001dded5e1145c167edcc59adc4d77a05ee202a039d75c2
Oct 30 19:18:48.997399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:48.997376    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf" event={"ID":"793048dc-32db-41ad-9964-04c275d02cbf","Type":"ContainerStarted","Data":"cc041c28fbf0e1f767ccd4ba45f05847317cef72331a7711a34978e7e8ce5838"}
Oct 30 19:18:49.053931 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:49.053901    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:49.208134 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:49.199069    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f5a0a6ec-2d19-4348-88f1-7370d80db8cf" path="/var/lib/kubelet/pods/f5a0a6ec-2d19-4348-88f1-7370d80db8cf/volumes"
Oct 30 19:18:49.226058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:49.225871    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"]
Oct 30 19:18:49.750358 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:49.750325    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:50.044115 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.043150    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf" event={"ID":"793048dc-32db-41ad-9964-04c275d02cbf","Type":"ContainerStarted","Data":"8dfe37d6425983618fd225a98a8621120318032baaccc38644649cee998d5ecb"}
Oct 30 19:18:50.048325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.047433    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf" event={"ID":"8db7e37a-9924-4ce9-bac0-67edd0374bf0","Type":"ContainerStarted","Data":"6f20d74cb970cbc0e05528997639e29d6516a60326647a6f3fd2684841eddce8"}
Oct 30 19:18:50.048325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.047492    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf" event={"ID":"8db7e37a-9924-4ce9-bac0-67edd0374bf0","Type":"ContainerStarted","Data":"a9cc2e6ed5db174326664b98cce9a6d20c73d920c097b1d5de689ee3c9bfb304"}
Oct 30 19:18:50.048325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.048265    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"
Oct 30 19:18:50.050732 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.050706    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb" event={"ID":"eb1c92a8-55e3-46fd-ae18-410645416414","Type":"ContainerStarted","Data":"7e7499637c0cdbda4f68f172120da8b49080f092289e59632692ffec45d67ab6"}
Oct 30 19:18:50.050927 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.050907    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb" event={"ID":"eb1c92a8-55e3-46fd-ae18-410645416414","Type":"ContainerStarted","Data":"ac7b29796e636f0e2001dded5e1145c167edcc59adc4d77a05ee202a039d75c2"}
Oct 30 19:18:50.052129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.051339    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:50.052129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.051427    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"
Oct 30 19:18:50.160314 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.160271    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf" podStartSLOduration=2.160257797 podStartE2EDuration="2.160257797s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:50.158556966 +0000 UTC m=+1903.104724869" watchObservedRunningTime="2026-10-30 19:18:50.160257797 +0000 UTC m=+1903.106425699"
Oct 30 19:18:50.274055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.271454    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb" podStartSLOduration=2.271436229 podStartE2EDuration="2.271436229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:50.268163035 +0000 UTC m=+1903.214330952" watchObservedRunningTime="2026-10-30 19:18:50.271436229 +0000 UTC m=+1903.217604134"
Oct 30 19:18:50.378155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.378043    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf" podStartSLOduration=2.378011478 podStartE2EDuration="2.378011478s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:50.377180028 +0000 UTC m=+1903.323347955" watchObservedRunningTime="2026-10-30 19:18:50.378011478 +0000 UTC m=+1903.324179382"
Oct 30 19:18:50.504434 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.500440    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:50.518085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.515669    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:50.586468 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.586427    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:50.854152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:50.854123    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:51.036574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.036539    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:51.058532 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.058499    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:51.059352 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.059333    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:18:51.136462 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.136388    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"]
Oct 30 19:18:51.140742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.140721    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.207673 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.207642    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"]
Oct 30 19:18:51.233657 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.233612    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62sc2\" (UniqueName: \"kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-kube-api-access-62sc2\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.233852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.233725    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.233852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.233785    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.233852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.233831    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.234062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.233977    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.234062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.234014    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.234185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.234134    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.234242 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.234180    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.234242 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.234223    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.335331 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.335299    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.335478 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.335398    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62sc2\" (UniqueName: \"kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-kube-api-access-62sc2\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.335478 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.335432    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.335585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.335481    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.335585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.335535    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.335683 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.335621    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.335683 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.335652    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.335787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.335697    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.335787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.335733    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.335882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.335788    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.335978 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.335956    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.336214 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.336189    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.336310 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.336241    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.336479 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.336460    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.337875 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.337853    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.337949 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.337935    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.393674 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.393611    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.393903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.393881    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62sc2\" (UniqueName: \"kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-kube-api-access-62sc2\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.456235 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.456212    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:51.634279 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.634246    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:51.885829 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.885805    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"]
Oct 30 19:18:51.887997 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:18:51.887970    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d980aa7_cbc1_4e29_b132_0245bfddd57b.slice/crio-5735d9d9907110ae7727c4ecfa100e16f98447b596539c10747b911f7e33cde2 WatchSource:0}: Error finding container 5735d9d9907110ae7727c4ecfa100e16f98447b596539c10747b911f7e33cde2: Status 404 returned error can't find the container with id 5735d9d9907110ae7727c4ecfa100e16f98447b596539c10747b911f7e33cde2
Oct 30 19:18:51.890469 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.890436    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:18:51.890531 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.890511    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:18:51.890572 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:51.890542    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:18:52.067176 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:52.067138    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf" event={"ID":"3d980aa7-cbc1-4e29-b132-0245bfddd57b","Type":"ContainerStarted","Data":"4fa705dc7a2071416f602dce5501325d9be8cbf9337a0b23d0adee2d92c67114"}
Oct 30 19:18:52.067649 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:52.067195    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf" event={"ID":"3d980aa7-cbc1-4e29-b132-0245bfddd57b","Type":"ContainerStarted","Data":"5735d9d9907110ae7727c4ecfa100e16f98447b596539c10747b911f7e33cde2"}
Oct 30 19:18:52.189874 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:52.189750    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf" podStartSLOduration=1.189731802 podStartE2EDuration="1.189731802s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:52.186970466 +0000 UTC m=+1905.133138374" watchObservedRunningTime="2026-10-30 19:18:52.189731802 +0000 UTC m=+1905.135899705"
Oct 30 19:18:52.458144 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:52.457883    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:52.461788 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:52.460705    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:18:52.461788 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:52.460769    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf" podUID="3d980aa7-cbc1-4e29-b132-0245bfddd57b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:18:52.985109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:52.985074    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:53.190980 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:53.190945    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:53.418465 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:53.418437    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:53.456989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:53.456955    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:18:53.457178 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:53.457011    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf" podUID="3d980aa7-cbc1-4e29-b132-0245bfddd57b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:18:53.844936 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:53.844862    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:54.050458 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:54.050427    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:54.457141 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:54.457111    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:18:54.457504 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:54.457162    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf" podUID="3d980aa7-cbc1-4e29-b132-0245bfddd57b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:18:55.456979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:55.456944    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:18:55.457159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:55.457000    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf" podUID="3d980aa7-cbc1-4e29-b132-0245bfddd57b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:18:55.917203 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:55.917005    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:56.190561 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.190474    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:56.389573 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.389436    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:56.461108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.458456    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:18:56.461108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.458519    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf" podUID="3d980aa7-cbc1-4e29-b132-0245bfddd57b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:18:56.574239 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.574192    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"]
Oct 30 19:18:56.579466 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.579444    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"
Oct 30 19:18:56.645979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.645926    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-4ec347\""
Oct 30 19:18:56.649141 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.649089    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"]
Oct 30 19:18:56.698452 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.698296    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-4ec347-dockercfg-s2j76\""
Oct 30 19:18:56.730102 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.729990    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e29dbede-923d-430c-895a-c28b4c89d0d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b\" (UID: \"e29dbede-923d-430c-895a-c28b4c89d0d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"
Oct 30 19:18:56.748215 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.748172    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flbdf\" (UniqueName: \"kubernetes.io/projected/e29dbede-923d-430c-895a-c28b4c89d0d7-kube-api-access-flbdf\") pod \"payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b\" (UID: \"e29dbede-923d-430c-895a-c28b4c89d0d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"
Oct 30 19:18:56.864434 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.859966    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e29dbede-923d-430c-895a-c28b4c89d0d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b\" (UID: \"e29dbede-923d-430c-895a-c28b4c89d0d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"
Oct 30 19:18:56.864434 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.860075    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flbdf\" (UniqueName: \"kubernetes.io/projected/e29dbede-923d-430c-895a-c28b4c89d0d7-kube-api-access-flbdf\") pod \"payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b\" (UID: \"e29dbede-923d-430c-895a-c28b4c89d0d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"
Oct 30 19:18:56.864434 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.861446    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e29dbede-923d-430c-895a-c28b4c89d0d7-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b\" (UID: \"e29dbede-923d-430c-895a-c28b4c89d0d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"
Oct 30 19:18:56.874586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.872325    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"]
Oct 30 19:18:56.881352 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.881329    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"]
Oct 30 19:18:56.881524 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.881414    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"
Oct 30 19:18:56.918076 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.918028    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-flbdf\" (UniqueName: \"kubernetes.io/projected/e29dbede-923d-430c-895a-c28b4c89d0d7-kube-api-access-flbdf\") pod \"payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b\" (UID: \"e29dbede-923d-430c-895a-c28b4c89d0d7\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"
Oct 30 19:18:56.961924 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.961890    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/930f1ac9-f21e-4f2a-ae60-43b41309469b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992\" (UID: \"930f1ac9-f21e-4f2a-ae60-43b41309469b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"
Oct 30 19:18:56.962162 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:56.962102    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fln9k\" (UniqueName: \"kubernetes.io/projected/930f1ac9-f21e-4f2a-ae60-43b41309469b-kube-api-access-fln9k\") pod \"payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992\" (UID: \"930f1ac9-f21e-4f2a-ae60-43b41309469b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"
Oct 30 19:18:57.063658 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:57.063614    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fln9k\" (UniqueName: \"kubernetes.io/projected/930f1ac9-f21e-4f2a-ae60-43b41309469b-kube-api-access-fln9k\") pod \"payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992\" (UID: \"930f1ac9-f21e-4f2a-ae60-43b41309469b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"
Oct 30 19:18:57.063854 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:57.063787    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/930f1ac9-f21e-4f2a-ae60-43b41309469b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992\" (UID: \"930f1ac9-f21e-4f2a-ae60-43b41309469b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"
Oct 30 19:18:57.064586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:57.064560    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/930f1ac9-f21e-4f2a-ae60-43b41309469b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992\" (UID: \"930f1ac9-f21e-4f2a-ae60-43b41309469b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"
Oct 30 19:18:57.144101 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:57.134290    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fln9k\" (UniqueName: \"kubernetes.io/projected/930f1ac9-f21e-4f2a-ae60-43b41309469b-kube-api-access-fln9k\") pod \"payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992\" (UID: \"930f1ac9-f21e-4f2a-ae60-43b41309469b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"
Oct 30 19:18:57.194786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:57.194755    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"
Oct 30 19:18:57.196897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:57.196867    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"
Oct 30 19:18:57.457575 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:57.457495    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused" start-of-body=
Oct 30 19:18:57.457575 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:57.457555    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf" podUID="3d980aa7-cbc1-4e29-b132-0245bfddd57b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.99:15021/healthz/ready\": dial tcp 10.134.0.99:15021: connect: connection refused"
Oct 30 19:18:57.672820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:57.672796    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"]
Oct 30 19:18:57.674971 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:18:57.674939    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode29dbede_923d_430c_895a_c28b4c89d0d7.slice/crio-827071c1860d03fc381d3ef55941b1c28217b6fafeac70cf1aadb88d3b0fb39f WatchSource:0}: Error finding container 827071c1860d03fc381d3ef55941b1c28217b6fafeac70cf1aadb88d3b0fb39f: Status 404 returned error can't find the container with id 827071c1860d03fc381d3ef55941b1c28217b6fafeac70cf1aadb88d3b0fb39f
Oct 30 19:18:57.717865 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:57.717839    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"]
Oct 30 19:18:57.718874 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:18:57.718834    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod930f1ac9_f21e_4f2a_ae60_43b41309469b.slice/crio-57f8e3cd52030a1624524282a9384c597db21e4191848aa4e3b234df9277b583 WatchSource:0}: Error finding container 57f8e3cd52030a1624524282a9384c597db21e4191848aa4e3b234df9277b583: Status 404 returned error can't find the container with id 57f8e3cd52030a1624524282a9384c597db21e4191848aa4e3b234df9277b583
Oct 30 19:18:58.113788 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:58.113749    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992" event={"ID":"930f1ac9-f21e-4f2a-ae60-43b41309469b","Type":"ContainerStarted","Data":"d3af5764f86b027af0209d6642903e65b4f802b959c652a58183b4deeca8f047"}
Oct 30 19:18:58.113979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:58.113824    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"
Oct 30 19:18:58.113979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:58.113851    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992" event={"ID":"930f1ac9-f21e-4f2a-ae60-43b41309469b","Type":"ContainerStarted","Data":"57f8e3cd52030a1624524282a9384c597db21e4191848aa4e3b234df9277b583"}
Oct 30 19:18:58.115672 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:58.115643    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b" event={"ID":"e29dbede-923d-430c-895a-c28b4c89d0d7","Type":"ContainerStarted","Data":"4d63ef3a72f6e3519440e0e696e31d180758f23b3d030839c67fc83b99fbf102"}
Oct 30 19:18:58.115813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:58.115696    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b" event={"ID":"e29dbede-923d-430c-895a-c28b4c89d0d7","Type":"ContainerStarted","Data":"827071c1860d03fc381d3ef55941b1c28217b6fafeac70cf1aadb88d3b0fb39f"}
Oct 30 19:18:58.115813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:58.115729    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"
Oct 30 19:18:58.224524 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:58.224471    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992" podStartSLOduration=2.224458143 podStartE2EDuration="2.224458143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:58.222685334 +0000 UTC m=+1911.168853236" watchObservedRunningTime="2026-10-30 19:18:58.224458143 +0000 UTC m=+1911.170626063"
Oct 30 19:18:58.334059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:58.333727    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b" podStartSLOduration=2.333707477 podStartE2EDuration="2.333707477s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:18:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:18:58.331103035 +0000 UTC m=+1911.277270933" watchObservedRunningTime="2026-10-30 19:18:58.333707477 +0000 UTC m=+1911.279875380"
Oct 30 19:18:58.525953 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:58.523394    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:58.525953 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:58.524618    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:58.530857 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:58.529575    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:18:59.298579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:59.298543    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:59.489566 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:59.489530    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:59.645079 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:59.644958    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"]
Oct 30 19:18:59.686836 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:59.686802    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:18:59.744374 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:18:59.744340    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"]
Oct 30 19:19:00.043962 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.043931    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:00.245178 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.243224    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:00.254078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.249449    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"]
Oct 30 19:19:00.254078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.251603    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st" podUID="295e1b69-00f7-4a36-b4fb-43bf138919e1" containerName="maas-api" containerID="cri-o://8a5feb2db4298feba8b90d4ca81e9d144b27c1a7d0b4fdcd4926fda45538c871" gracePeriod=30
Oct 30 19:19:00.458914 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.453595    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:00.578525 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.578501    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:19:00.679120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.673490    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-metrics-certs\" (UniqueName: \"kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-metrics-certs\") pod \"295e1b69-00f7-4a36-b4fb-43bf138919e1\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") "
Oct 30 19:19:00.679120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.673602    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/295e1b69-00f7-4a36-b4fb-43bf138919e1-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/295e1b69-00f7-4a36-b4fb-43bf138919e1-openshift-service-ca\") pod \"295e1b69-00f7-4a36-b4fb-43bf138919e1\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") "
Oct 30 19:19:00.679120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.673657    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-maas-api-tls\") pod \"295e1b69-00f7-4a36-b4fb-43bf138919e1\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") "
Oct 30 19:19:00.679120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.673813    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/295e1b69-00f7-4a36-b4fb-43bf138919e1-kube-api-access-lfjdw\" (UniqueName: \"kubernetes.io/projected/295e1b69-00f7-4a36-b4fb-43bf138919e1-kube-api-access-lfjdw\") pod \"295e1b69-00f7-4a36-b4fb-43bf138919e1\" (UID: \"295e1b69-00f7-4a36-b4fb-43bf138919e1\") "
Oct 30 19:19:00.683062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.681529    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/295e1b69-00f7-4a36-b4fb-43bf138919e1-openshift-service-ca" pod "295e1b69-00f7-4a36-b4fb-43bf138919e1" (UID: "295e1b69-00f7-4a36-b4fb-43bf138919e1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:19:00.686485 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.686432    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-maas-api-tls" pod "295e1b69-00f7-4a36-b4fb-43bf138919e1" (UID: "295e1b69-00f7-4a36-b4fb-43bf138919e1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:19:00.750396 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.727541    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-metrics-certs" pod "295e1b69-00f7-4a36-b4fb-43bf138919e1" (UID: "295e1b69-00f7-4a36-b4fb-43bf138919e1"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:19:00.750396 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.731538    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/295e1b69-00f7-4a36-b4fb-43bf138919e1-kube-api-access-lfjdw" pod "295e1b69-00f7-4a36-b4fb-43bf138919e1" (UID: "295e1b69-00f7-4a36-b4fb-43bf138919e1"). InnerVolumeSpecName "kube-api-access-lfjdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:19:00.775673 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.775648    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:00.775673 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.775674    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/295e1b69-00f7-4a36-b4fb-43bf138919e1-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:00.775870 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.775688    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/295e1b69-00f7-4a36-b4fb-43bf138919e1-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:00.775870 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:00.775700    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lfjdw\" (UniqueName: \"kubernetes.io/projected/295e1b69-00f7-4a36-b4fb-43bf138919e1-kube-api-access-lfjdw\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:01.061631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:01.061597    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"
Oct 30 19:19:01.175987 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:01.175945    5704 generic.go:356] "Generic (PLEG): container finished" podID="295e1b69-00f7-4a36-b4fb-43bf138919e1" containerID="8a5feb2db4298feba8b90d4ca81e9d144b27c1a7d0b4fdcd4926fda45538c871" exitCode=0
Oct 30 19:19:01.176221 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:01.176106    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"
Oct 30 19:19:01.189774 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:01.189742    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st" event={"ID":"295e1b69-00f7-4a36-b4fb-43bf138919e1","Type":"ContainerDied","Data":"8a5feb2db4298feba8b90d4ca81e9d144b27c1a7d0b4fdcd4926fda45538c871"}
Oct 30 19:19:01.189903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:01.189790    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st" event={"ID":"295e1b69-00f7-4a36-b4fb-43bf138919e1","Type":"ContainerDied","Data":"6b24d3df51631a628bc174256bfac53ffa1f6732714474a44299ef912f2472c4"}
Oct 30 19:19:01.189903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:01.189812    5704 scope.go:122] "RemoveContainer" containerID="8a5feb2db4298feba8b90d4ca81e9d144b27c1a7d0b4fdcd4926fda45538c871"
Oct 30 19:19:01.203114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:01.203090    5704 scope.go:122] "RemoveContainer" containerID="8a5feb2db4298feba8b90d4ca81e9d144b27c1a7d0b4fdcd4926fda45538c871"
Oct 30 19:19:01.203437 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:19:01.203401    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a5feb2db4298feba8b90d4ca81e9d144b27c1a7d0b4fdcd4926fda45538c871\": container with ID starting with 8a5feb2db4298feba8b90d4ca81e9d144b27c1a7d0b4fdcd4926fda45538c871 not found: ID does not exist" containerID="8a5feb2db4298feba8b90d4ca81e9d144b27c1a7d0b4fdcd4926fda45538c871"
Oct 30 19:19:01.203557 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:01.203445    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a5feb2db4298feba8b90d4ca81e9d144b27c1a7d0b4fdcd4926fda45538c871"} err="failed to get container status \"8a5feb2db4298feba8b90d4ca81e9d144b27c1a7d0b4fdcd4926fda45538c871\": rpc error: code = NotFound desc = could not find container \"8a5feb2db4298feba8b90d4ca81e9d144b27c1a7d0b4fdcd4926fda45538c871\": container with ID starting with 8a5feb2db4298feba8b90d4ca81e9d144b27c1a7d0b4fdcd4926fda45538c871 not found: ID does not exist"
Oct 30 19:19:01.346177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:01.346089    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"]
Oct 30 19:19:01.355143 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:01.351896    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-942c5e85-79bb7889c6-ch8st"]
Oct 30 19:19:02.070391 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:02.070365    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"
Oct 30 19:19:02.232181 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:02.232153    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:03.180276 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:03.180249    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="295e1b69-00f7-4a36-b4fb-43bf138919e1" path="/var/lib/kubelet/pods/295e1b69-00f7-4a36-b4fb-43bf138919e1/volumes"
Oct 30 19:19:03.383439 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:19:03.383405    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:19:04.516260 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:04.516229    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:04.650163 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:04.650118    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d" podUID="b2d70f4d-f3d6-400b-8eec-0d444b90b72b" containerName="payload-processing" containerID="cri-o://85833ff32f7069a992a303e0b9adde86bb673cd640157b18ae88fdbda5fe9537" gracePeriod=25
Oct 30 19:19:04.745140 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:04.745075    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv" podUID="08aeb70b-bc30-42c0-871c-89afcd4a34e3" containerName="payload-pre-processing" containerID="cri-o://7c4e70876d7c6383d4d0e23f7bef829dc94f724318beaed593c4e60253457dd3" gracePeriod=25
Oct 30 19:19:04.747187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:04.747152    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:04.900457 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:04.900428    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"]
Oct 30 19:19:04.901219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:04.901194    5704 cpu_manager.go:414] "containerMap: removing container" podUID="295e1b69-00f7-4a36-b4fb-43bf138919e1" containerName="maas-api"
Oct 30 19:19:04.901338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:04.901220    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="295e1b69-00f7-4a36-b4fb-43bf138919e1" containerName="maas-api"
Oct 30 19:19:04.901399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:04.901385    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="295e1b69-00f7-4a36-b4fb-43bf138919e1" containerName="maas-api"
Oct 30 19:19:04.910065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:04.908407    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"
Oct 30 19:19:04.969495 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:04.967851    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"]
Oct 30 19:19:05.126226 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.123636    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r\" (UID: \"298df6c6-b7d1-46e2-bbc9-09a000ae3f30\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"
Oct 30 19:19:05.126226 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.123764    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qddd6\" (UniqueName: \"kubernetes.io/projected/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-kube-api-access-qddd6\") pod \"payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r\" (UID: \"298df6c6-b7d1-46e2-bbc9-09a000ae3f30\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"
Oct 30 19:19:05.126226 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.124091    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-fb4a1b\""
Oct 30 19:19:05.126226 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.124207    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-fb4a1b-dockercfg-9nf2n\""
Oct 30 19:19:05.147090 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.137491    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:05.212134 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.212108    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"]
Oct 30 19:19:05.214104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.214083    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"
Oct 30 19:19:05.215454 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.215431    5704 generic.go:356] "Generic (PLEG): container finished" podID="b2d70f4d-f3d6-400b-8eec-0d444b90b72b" containerID="85833ff32f7069a992a303e0b9adde86bb673cd640157b18ae88fdbda5fe9537" exitCode=0
Oct 30 19:19:05.216249 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.216231    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"]
Oct 30 19:19:05.216338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.216298    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d" event={"ID":"b2d70f4d-f3d6-400b-8eec-0d444b90b72b","Type":"ContainerDied","Data":"85833ff32f7069a992a303e0b9adde86bb673cd640157b18ae88fdbda5fe9537"}
Oct 30 19:19:05.216338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.216318    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"
Oct 30 19:19:05.216926 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.216902    5704 generic.go:356] "Generic (PLEG): container finished" podID="08aeb70b-bc30-42c0-871c-89afcd4a34e3" containerID="7c4e70876d7c6383d4d0e23f7bef829dc94f724318beaed593c4e60253457dd3" exitCode=0
Oct 30 19:19:05.217020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.216926    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv" event={"ID":"08aeb70b-bc30-42c0-871c-89afcd4a34e3","Type":"ContainerDied","Data":"7c4e70876d7c6383d4d0e23f7bef829dc94f724318beaed593c4e60253457dd3"}
Oct 30 19:19:05.217020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.216950    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"
Oct 30 19:19:05.217020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.216969    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv" event={"ID":"08aeb70b-bc30-42c0-871c-89afcd4a34e3","Type":"ContainerDied","Data":"2b2829af700f55aedbace6e61eafa58a6707a469b4af433c68a742ec275714c5"}
Oct 30 19:19:05.217020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.216997    5704 scope.go:122] "RemoveContainer" containerID="7c4e70876d7c6383d4d0e23f7bef829dc94f724318beaed593c4e60253457dd3"
Oct 30 19:19:05.225199 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.225172    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r\" (UID: \"298df6c6-b7d1-46e2-bbc9-09a000ae3f30\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"
Oct 30 19:19:05.225325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.225261    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qddd6\" (UniqueName: \"kubernetes.io/projected/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-kube-api-access-qddd6\") pod \"payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r\" (UID: \"298df6c6-b7d1-46e2-bbc9-09a000ae3f30\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"
Oct 30 19:19:05.227439 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.227414    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r\" (UID: \"298df6c6-b7d1-46e2-bbc9-09a000ae3f30\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"
Oct 30 19:19:05.231508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.231488    5704 scope.go:122] "RemoveContainer" containerID="7c4e70876d7c6383d4d0e23f7bef829dc94f724318beaed593c4e60253457dd3"
Oct 30 19:19:05.233176 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:19:05.232870    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c4e70876d7c6383d4d0e23f7bef829dc94f724318beaed593c4e60253457dd3\": container with ID starting with 7c4e70876d7c6383d4d0e23f7bef829dc94f724318beaed593c4e60253457dd3 not found: ID does not exist" containerID="7c4e70876d7c6383d4d0e23f7bef829dc94f724318beaed593c4e60253457dd3"
Oct 30 19:19:05.233176 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.232910    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c4e70876d7c6383d4d0e23f7bef829dc94f724318beaed593c4e60253457dd3"} err="failed to get container status \"7c4e70876d7c6383d4d0e23f7bef829dc94f724318beaed593c4e60253457dd3\": rpc error: code = NotFound desc = could not find container \"7c4e70876d7c6383d4d0e23f7bef829dc94f724318beaed593c4e60253457dd3\": container with ID starting with 7c4e70876d7c6383d4d0e23f7bef829dc94f724318beaed593c4e60253457dd3 not found: ID does not exist"
Oct 30 19:19:05.243753 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.243726    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"
Oct 30 19:19:05.282890 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.282859    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qddd6\" (UniqueName: \"kubernetes.io/projected/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-kube-api-access-qddd6\") pod \"payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r\" (UID: \"298df6c6-b7d1-46e2-bbc9-09a000ae3f30\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"
Oct 30 19:19:05.326005 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.325975    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-plugins-config-volume\") pod \"b2d70f4d-f3d6-400b-8eec-0d444b90b72b\" (UID: \"b2d70f4d-f3d6-400b-8eec-0d444b90b72b\") "
Oct 30 19:19:05.326245 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.326228    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/08aeb70b-bc30-42c0-871c-89afcd4a34e3-kube-api-access-npr6g\" (UniqueName: \"kubernetes.io/projected/08aeb70b-bc30-42c0-871c-89afcd4a34e3-kube-api-access-npr6g\") pod \"08aeb70b-bc30-42c0-871c-89afcd4a34e3\" (UID: \"08aeb70b-bc30-42c0-871c-89afcd4a34e3\") "
Oct 30 19:19:05.326292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.326257    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-kube-api-access-9gz9c\" (UniqueName: \"kubernetes.io/projected/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-kube-api-access-9gz9c\") pod \"b2d70f4d-f3d6-400b-8eec-0d444b90b72b\" (UID: \"b2d70f4d-f3d6-400b-8eec-0d444b90b72b\") "
Oct 30 19:19:05.326292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.326279    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/08aeb70b-bc30-42c0-871c-89afcd4a34e3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08aeb70b-bc30-42c0-871c-89afcd4a34e3-plugins-config-volume\") pod \"08aeb70b-bc30-42c0-871c-89afcd4a34e3\" (UID: \"08aeb70b-bc30-42c0-871c-89afcd4a34e3\") "
Oct 30 19:19:05.326378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.326292    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-plugins-config-volume" pod "b2d70f4d-f3d6-400b-8eec-0d444b90b72b" (UID: "b2d70f4d-f3d6-400b-8eec-0d444b90b72b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:19:05.326519 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.326494    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf\" (UID: \"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"
Oct 30 19:19:05.326653 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.326632    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv9qc\" (UniqueName: \"kubernetes.io/projected/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-kube-api-access-sv9qc\") pod \"payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf\" (UID: \"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"
Oct 30 19:19:05.326724 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.326699    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08aeb70b-bc30-42c0-871c-89afcd4a34e3-plugins-config-volume" pod "08aeb70b-bc30-42c0-871c-89afcd4a34e3" (UID: "08aeb70b-bc30-42c0-871c-89afcd4a34e3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:19:05.326882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.326862    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/08aeb70b-bc30-42c0-871c-89afcd4a34e3-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:05.326950 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.326883    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:05.328459 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.328432    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08aeb70b-bc30-42c0-871c-89afcd4a34e3-kube-api-access-npr6g" pod "08aeb70b-bc30-42c0-871c-89afcd4a34e3" (UID: "08aeb70b-bc30-42c0-871c-89afcd4a34e3"). InnerVolumeSpecName "kube-api-access-npr6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:19:05.328648 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.328627    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-kube-api-access-9gz9c" pod "b2d70f4d-f3d6-400b-8eec-0d444b90b72b" (UID: "b2d70f4d-f3d6-400b-8eec-0d444b90b72b"). InnerVolumeSpecName "kube-api-access-9gz9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:19:05.427940 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.427856    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf\" (UID: \"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"
Oct 30 19:19:05.427940 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.427922    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv9qc\" (UniqueName: \"kubernetes.io/projected/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-kube-api-access-sv9qc\") pod \"payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf\" (UID: \"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"
Oct 30 19:19:05.428216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.428018    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npr6g\" (UniqueName: \"kubernetes.io/projected/08aeb70b-bc30-42c0-871c-89afcd4a34e3-kube-api-access-npr6g\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:05.428216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.428029    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gz9c\" (UniqueName: \"kubernetes.io/projected/b2d70f4d-f3d6-400b-8eec-0d444b90b72b-kube-api-access-9gz9c\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:05.428655 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.428628    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf\" (UID: \"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"
Oct 30 19:19:05.484886 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.484859    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv9qc\" (UniqueName: \"kubernetes.io/projected/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-kube-api-access-sv9qc\") pod \"payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf\" (UID: \"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"
Oct 30 19:19:05.528197 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.528176    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"
Oct 30 19:19:05.537367 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.537348    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"
Oct 30 19:19:05.694985 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.694909    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"]
Oct 30 19:19:05.699137 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.699110    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-942c5e85-5949468747-jdcbv"]
Oct 30 19:19:05.906787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.906757    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"]
Oct 30 19:19:05.908892 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:19:05.908862    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod298df6c6_b7d1_46e2_bbc9_09a000ae3f30.slice/crio-aea3d384cabdea84887da9383dcec170318533cfd57fc2d964690b582cccf2bc WatchSource:0}: Error finding container aea3d384cabdea84887da9383dcec170318533cfd57fc2d964690b582cccf2bc: Status 404 returned error can't find the container with id aea3d384cabdea84887da9383dcec170318533cfd57fc2d964690b582cccf2bc
Oct 30 19:19:05.954758 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:05.954516    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"]
Oct 30 19:19:05.955588 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:19:05.955566    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda933f15e_0fa7_4c6d_99a2_b3c4ba0250da.slice/crio-479618d8a8248d1a2e33957f02ffec691feb33a11d9c8d8a15787d820549f3e6 WatchSource:0}: Error finding container 479618d8a8248d1a2e33957f02ffec691feb33a11d9c8d8a15787d820549f3e6: Status 404 returned error can't find the container with id 479618d8a8248d1a2e33957f02ffec691feb33a11d9c8d8a15787d820549f3e6
Oct 30 19:19:06.224852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.224774    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r" event={"ID":"298df6c6-b7d1-46e2-bbc9-09a000ae3f30","Type":"ContainerStarted","Data":"e7de5fe6ade0ce20b5d647f072cf7a3ea21697cb1b1793cef40e963df2a7f19f"}
Oct 30 19:19:06.224852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.224828    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r" event={"ID":"298df6c6-b7d1-46e2-bbc9-09a000ae3f30","Type":"ContainerStarted","Data":"aea3d384cabdea84887da9383dcec170318533cfd57fc2d964690b582cccf2bc"}
Oct 30 19:19:06.225104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.224853    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"
Oct 30 19:19:06.226397 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.226372    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf" event={"ID":"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da","Type":"ContainerStarted","Data":"09c3cbfdcec3d7a9720b58d695b1da75bf8dd9c5dcd034a3636390478203a680"}
Oct 30 19:19:06.226529 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.226418    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf" event={"ID":"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da","Type":"ContainerStarted","Data":"479618d8a8248d1a2e33957f02ffec691feb33a11d9c8d8a15787d820549f3e6"}
Oct 30 19:19:06.226529 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.226447    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"
Oct 30 19:19:06.227763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.227746    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"
Oct 30 19:19:06.227763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.227758    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d" event={"ID":"b2d70f4d-f3d6-400b-8eec-0d444b90b72b","Type":"ContainerDied","Data":"cb5c68df7b9e6c17c37bb5e41c7e1cf4de3eaae7002c4a9ebd5c902ac3f40d1f"}
Oct 30 19:19:06.227906 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.227808    5704 scope.go:122] "RemoveContainer" containerID="85833ff32f7069a992a303e0b9adde86bb673cd640157b18ae88fdbda5fe9537"
Oct 30 19:19:06.286660 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.286638    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"]
Oct 30 19:19:06.287706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.287680    5704 cpu_manager.go:414] "containerMap: removing container" podUID="b2d70f4d-f3d6-400b-8eec-0d444b90b72b" containerName="payload-processing"
Oct 30 19:19:06.287706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.287705    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b2d70f4d-f3d6-400b-8eec-0d444b90b72b" containerName="payload-processing"
Oct 30 19:19:06.287839 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.287740    5704 cpu_manager.go:414] "containerMap: removing container" podUID="08aeb70b-bc30-42c0-871c-89afcd4a34e3" containerName="payload-pre-processing"
Oct 30 19:19:06.287839 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.287749    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="08aeb70b-bc30-42c0-871c-89afcd4a34e3" containerName="payload-pre-processing"
Oct 30 19:19:06.287938 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.287925    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="b2d70f4d-f3d6-400b-8eec-0d444b90b72b" containerName="payload-processing"
Oct 30 19:19:06.287977 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.287946    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="08aeb70b-bc30-42c0-871c-89afcd4a34e3" containerName="payload-pre-processing"
Oct 30 19:19:06.292798 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.292783    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.338473 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.338439    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.338620 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.338496    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66bc2818-931a-410d-a3ad-50ad679764c4-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.338697 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.338625    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.338697 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.338661    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.338814 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.338764    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.338901 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.338862    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.338964 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.338947    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.339021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.338989    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd9zz\" (UniqueName: \"kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-kube-api-access-wd9zz\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.339115 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.339068    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66bc2818-931a-410d-a3ad-50ad679764c4-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.340552 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.340507    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r" podStartSLOduration=2.340495133 podStartE2EDuration="2.340495133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:06.337819691 +0000 UTC m=+1919.283987595" watchObservedRunningTime="2026-10-30 19:19:06.340495133 +0000 UTC m=+1919.286663036"
Oct 30 19:19:06.379692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.379663    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"]
Oct 30 19:19:06.440236 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.440206    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.440382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.440239    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66bc2818-931a-410d-a3ad-50ad679764c4-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.440382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.440261    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.440382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.440306    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.440382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.440360    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.440577 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.440440    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.440577 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.440502    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.440577 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.440546    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd9zz\" (UniqueName: \"kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-kube-api-access-wd9zz\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.440731 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.440591    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66bc2818-931a-410d-a3ad-50ad679764c4-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.440731 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.440613    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.440731 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.440702    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-data\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.440876 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.440761    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-credential-socket\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.440928 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.440882    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-certs\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.441311 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.441283    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66bc2818-931a-410d-a3ad-50ad679764c4-istiod-ca-cert\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.443064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.443007    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-envoy\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.443295 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.443274    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66bc2818-931a-410d-a3ad-50ad679764c4-istio-podinfo\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.444765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.444747    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"]
Oct 30 19:19:06.453567 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.453519    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-942c5e85-695f57fd6b-2f29d"]
Oct 30 19:19:06.509626 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.509592    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-istio-token\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.511657 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.511631    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd9zz\" (UniqueName: \"kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-kube-api-access-wd9zz\") pod \"e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") " pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.606765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.606735    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:06.678203 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.677824    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf" podStartSLOduration=1.677802727 podStartE2EDuration="1.677802727s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:06.673807264 +0000 UTC m=+1919.619975168" watchObservedRunningTime="2026-10-30 19:19:06.677802727 +0000 UTC m=+1919.623970631"
Oct 30 19:19:06.984914 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.984889    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"]
Oct 30 19:19:06.987674 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:19:06.987647    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66bc2818_931a_410d_a3ad_50ad679764c4.slice/crio-8621cd897809e228859cfb958b33494c647ce2631487d9b92d972737044522ba WatchSource:0}: Error finding container 8621cd897809e228859cfb958b33494c647ce2631487d9b92d972737044522ba: Status 404 returned error can't find the container with id 8621cd897809e228859cfb958b33494c647ce2631487d9b92d972737044522ba
Oct 30 19:19:06.990312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.990280    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:06.990398 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.990343    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:06.990398 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:06.990375    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:19:07.182939 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:07.182911    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="08aeb70b-bc30-42c0-871c-89afcd4a34e3" path="/var/lib/kubelet/pods/08aeb70b-bc30-42c0-871c-89afcd4a34e3/volumes"
Oct 30 19:19:07.183349 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:07.183331    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b2d70f4d-f3d6-400b-8eec-0d444b90b72b" path="/var/lib/kubelet/pods/b2d70f4d-f3d6-400b-8eec-0d444b90b72b/volumes"
Oct 30 19:19:07.289830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:07.289785    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r" event={"ID":"66bc2818-931a-410d-a3ad-50ad679764c4","Type":"ContainerStarted","Data":"86fc574629fa3addf4cfc29b9290b664a83241c61ec1efed8b7bf8f25ed8e4ad"}
Oct 30 19:19:07.290044 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:07.289884    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r" event={"ID":"66bc2818-931a-410d-a3ad-50ad679764c4","Type":"ContainerStarted","Data":"8621cd897809e228859cfb958b33494c647ce2631487d9b92d972737044522ba"}
Oct 30 19:19:07.295361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:07.295331    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:07.406650 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:07.406597    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r" podStartSLOduration=1.406583345 podStartE2EDuration="1.406583345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:07.403659585 +0000 UTC m=+1920.349827500" watchObservedRunningTime="2026-10-30 19:19:07.406583345 +0000 UTC m=+1920.352751249"
Oct 30 19:19:07.606985 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:07.606965    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:07.751061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:07.749879    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:08.053197 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:08.049874    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:08.608878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:08.608847    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:19:08.609284 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:08.608902    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r" podUID="66bc2818-931a-410d-a3ad-50ad679764c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 19:19:08.718167 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:08.718122    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:09.144946 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:09.144906    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"
Oct 30 19:19:09.145177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:09.145017    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"
Oct 30 19:19:09.607092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:09.607056    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:19:09.607277 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:09.607113    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r" podUID="66bc2818-931a-410d-a3ad-50ad679764c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 19:19:10.607268 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:10.607233    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:19:10.607686 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:10.607288    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r" podUID="66bc2818-931a-410d-a3ad-50ad679764c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 19:19:11.607419 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:11.607384    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:19:11.607826 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:11.607431    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r" podUID="66bc2818-931a-410d-a3ad-50ad679764c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 19:19:11.853395 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:11.853349    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"]
Oct 30 19:19:12.261056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:12.252201    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:12.448217 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:12.447747    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:12.616850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:12.616446    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused" start-of-body=
Oct 30 19:19:12.616850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:12.616507    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r" podUID="66bc2818-931a-410d-a3ad-50ad679764c4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.104:15021/healthz/ready\": dial tcp 10.134.0.104:15021: connect: connection refused"
Oct 30 19:19:12.638559 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:12.638520    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:13.741123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:13.741079    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:13.838481 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:13.838440    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:13.838776 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:13.838754    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:13.839772 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:13.839750    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:19:16.854471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:16.854411    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7" podUID="ecd45fb1-3b71-4953-936f-1aa47f8eebd5" containerName="payload-processing" containerID="cri-o://2dd8ab300a0fa17adbe8aee3836ea584aeb87ac56980f389f4665ff2eac45492" gracePeriod=25
Oct 30 19:19:17.124949 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.124918    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"
Oct 30 19:19:17.230850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.230823    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-plugins-config-volume\") pod \"ecd45fb1-3b71-4953-936f-1aa47f8eebd5\" (UID: \"ecd45fb1-3b71-4953-936f-1aa47f8eebd5\") "
Oct 30 19:19:17.231000 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.230966    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-kube-api-access-dmqlm\" (UniqueName: \"kubernetes.io/projected/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-kube-api-access-dmqlm\") pod \"ecd45fb1-3b71-4953-936f-1aa47f8eebd5\" (UID: \"ecd45fb1-3b71-4953-936f-1aa47f8eebd5\") "
Oct 30 19:19:17.231192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.231170    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-plugins-config-volume" pod "ecd45fb1-3b71-4953-936f-1aa47f8eebd5" (UID: "ecd45fb1-3b71-4953-936f-1aa47f8eebd5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:19:17.231302 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.231287    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:17.232976 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.232952    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-kube-api-access-dmqlm" pod "ecd45fb1-3b71-4953-936f-1aa47f8eebd5" (UID: "ecd45fb1-3b71-4953-936f-1aa47f8eebd5"). InnerVolumeSpecName "kube-api-access-dmqlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:19:17.292512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.292490    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"
Oct 30 19:19:17.292640 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.292557    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"
Oct 30 19:19:17.333219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.333187    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmqlm\" (UniqueName: \"kubernetes.io/projected/ecd45fb1-3b71-4953-936f-1aa47f8eebd5-kube-api-access-dmqlm\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:19:17.478825 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.478734    5704 generic.go:356] "Generic (PLEG): container finished" podID="ecd45fb1-3b71-4953-936f-1aa47f8eebd5" containerID="2dd8ab300a0fa17adbe8aee3836ea584aeb87ac56980f389f4665ff2eac45492" exitCode=0
Oct 30 19:19:17.478825 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.478783    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"
Oct 30 19:19:17.479003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.478819    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7" event={"ID":"ecd45fb1-3b71-4953-936f-1aa47f8eebd5","Type":"ContainerDied","Data":"2dd8ab300a0fa17adbe8aee3836ea584aeb87ac56980f389f4665ff2eac45492"}
Oct 30 19:19:17.479003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.478887    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7" event={"ID":"ecd45fb1-3b71-4953-936f-1aa47f8eebd5","Type":"ContainerDied","Data":"ff29fb77ab4bbc891d3685a289093545e5542fa1efa53f4386e5c9f03cfead60"}
Oct 30 19:19:17.479003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.478920    5704 scope.go:122] "RemoveContainer" containerID="2dd8ab300a0fa17adbe8aee3836ea584aeb87ac56980f389f4665ff2eac45492"
Oct 30 19:19:17.489423 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.489407    5704 scope.go:122] "RemoveContainer" containerID="2dd8ab300a0fa17adbe8aee3836ea584aeb87ac56980f389f4665ff2eac45492"
Oct 30 19:19:17.489669 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:19:17.489642    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2dd8ab300a0fa17adbe8aee3836ea584aeb87ac56980f389f4665ff2eac45492\": container with ID starting with 2dd8ab300a0fa17adbe8aee3836ea584aeb87ac56980f389f4665ff2eac45492 not found: ID does not exist" containerID="2dd8ab300a0fa17adbe8aee3836ea584aeb87ac56980f389f4665ff2eac45492"
Oct 30 19:19:17.489736 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.489665    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2dd8ab300a0fa17adbe8aee3836ea584aeb87ac56980f389f4665ff2eac45492"} err="failed to get container status \"2dd8ab300a0fa17adbe8aee3836ea584aeb87ac56980f389f4665ff2eac45492\": rpc error: code = NotFound desc = could not find container \"2dd8ab300a0fa17adbe8aee3836ea584aeb87ac56980f389f4665ff2eac45492\": container with ID starting with 2dd8ab300a0fa17adbe8aee3836ea584aeb87ac56980f389f4665ff2eac45492 not found: ID does not exist"
Oct 30 19:19:17.619559 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.619524    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"]
Oct 30 19:19:17.625869 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.625844    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-2ec7ea-65f65c57d-rlhx7"]
Oct 30 19:19:17.671063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:17.667971    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:19.180845 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:19.180815    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ecd45fb1-3b71-4953-936f-1aa47f8eebd5" path="/var/lib/kubelet/pods/ecd45fb1-3b71-4953-936f-1aa47f8eebd5/volumes"
Oct 30 19:19:21.909997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:21.909963    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"]
Oct 30 19:19:21.910598 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:21.910584    5704 cpu_manager.go:414] "containerMap: removing container" podUID="ecd45fb1-3b71-4953-936f-1aa47f8eebd5" containerName="payload-processing"
Oct 30 19:19:21.910656 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:21.910599    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ecd45fb1-3b71-4953-936f-1aa47f8eebd5" containerName="payload-processing"
Oct 30 19:19:21.910730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:21.910721    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="ecd45fb1-3b71-4953-936f-1aa47f8eebd5" containerName="payload-processing"
Oct 30 19:19:21.914633 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:21.914614    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:21.963163 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:21.963138    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-905566-serving-cert\""
Oct 30 19:19:21.973749 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:21.973727    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"]
Oct 30 19:19:22.092372 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.092341    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-maas-api-tls\") pod \"maas-api-e2e-ipp-a-905566-58c8b66f64-bs676\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:22.092583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.092379    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95061afb-d0ef-47d6-975e-fc5d7d500e16-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-905566-58c8b66f64-bs676\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:22.092583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.092495    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-metrics-certs\") pod \"maas-api-e2e-ipp-a-905566-58c8b66f64-bs676\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:22.092663 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.092626    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtdkf\" (UniqueName: \"kubernetes.io/projected/95061afb-d0ef-47d6-975e-fc5d7d500e16-kube-api-access-rtdkf\") pod \"maas-api-e2e-ipp-a-905566-58c8b66f64-bs676\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:22.193811 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.193724    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-maas-api-tls\") pod \"maas-api-e2e-ipp-a-905566-58c8b66f64-bs676\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:22.195151 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.195112    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95061afb-d0ef-47d6-975e-fc5d7d500e16-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-905566-58c8b66f64-bs676\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:22.195317 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.195220    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-metrics-certs\") pod \"maas-api-e2e-ipp-a-905566-58c8b66f64-bs676\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:22.195415 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.195394    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtdkf\" (UniqueName: \"kubernetes.io/projected/95061afb-d0ef-47d6-975e-fc5d7d500e16-kube-api-access-rtdkf\") pod \"maas-api-e2e-ipp-a-905566-58c8b66f64-bs676\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:22.196166 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:19:22.196137    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-serving-cert: secret "maas-api-e2e-ipp-a-905566-serving-cert" not found
Oct 30 19:19:22.197184 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:19:22.197163    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-maas-api-tls podName:95061afb-d0ef-47d6-975e-fc5d7d500e16 nodeName:}" failed. No retries permitted until 2026-10-30 19:19:22.697141363 +0000 UTC m=+1935.643309247 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-maas-api-tls") pod "maas-api-e2e-ipp-a-905566-58c8b66f64-bs676" (UID: "95061afb-d0ef-47d6-975e-fc5d7d500e16") : secret "maas-api-e2e-ipp-a-905566-serving-cert" not found
Oct 30 19:19:22.197411 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.197387    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95061afb-d0ef-47d6-975e-fc5d7d500e16-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-905566-58c8b66f64-bs676\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:22.203187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.203163    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-metrics-certs\") pod \"maas-api-e2e-ipp-a-905566-58c8b66f64-bs676\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:22.235729 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.235699    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:22.255653 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.255627    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtdkf\" (UniqueName: \"kubernetes.io/projected/95061afb-d0ef-47d6-975e-fc5d7d500e16-kube-api-access-rtdkf\") pod \"maas-api-e2e-ipp-a-905566-58c8b66f64-bs676\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:22.474609 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.474531    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:22.701008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.700978    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-maas-api-tls\") pod \"maas-api-e2e-ipp-a-905566-58c8b66f64-bs676\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:22.703374 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.703344    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-maas-api-tls\") pod \"maas-api-e2e-ipp-a-905566-58c8b66f64-bs676\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:22.828374 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:22.828344    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:23.201253 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:23.201231    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"]
Oct 30 19:19:23.204923 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:19:23.204895    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95061afb_d0ef_47d6_975e_fc5d7d500e16.slice/crio-ada40ace1345282b9fc78c6fa4a0f3018a36aed73cc52c0bbe91998bf48d825a WatchSource:0}: Error finding container ada40ace1345282b9fc78c6fa4a0f3018a36aed73cc52c0bbe91998bf48d825a: Status 404 returned error can't find the container with id ada40ace1345282b9fc78c6fa4a0f3018a36aed73cc52c0bbe91998bf48d825a
Oct 30 19:19:23.521929 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:23.521896    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676" event={"ID":"95061afb-d0ef-47d6-975e-fc5d7d500e16","Type":"ContainerStarted","Data":"ada40ace1345282b9fc78c6fa4a0f3018a36aed73cc52c0bbe91998bf48d825a"}
Oct 30 19:19:24.530116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:24.530072    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676" event={"ID":"95061afb-d0ef-47d6-975e-fc5d7d500e16","Type":"ContainerStarted","Data":"5a4e348f490a9876e7c16abd88c0cfdf49a0673c05aaf048a7984ceb77cb481c"}
Oct 30 19:19:24.530525 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:24.530176    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:24.652654 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:24.652604    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676" podStartSLOduration=2.830515823 podStartE2EDuration="3.652587365s" totalImagesPullingTime="822.071542ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:24.650210236 +0000 UTC m=+1937.596378139" watchObservedRunningTime="2026-10-30 19:19:24.652587365 +0000 UTC m=+1937.598755303"
Oct 30 19:19:26.958341 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:26.958298    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:27.506858 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:27.506828    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:19:30.541817 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:30.541789    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:19:31.717922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:31.717886    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:33.385088 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:19:33.385053    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:19:36.158043 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:36.158012    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:36.340690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:36.340643    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:36.521521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:36.521491    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:40.640352 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.640305    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"]
Oct 30 19:19:40.644378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.644357    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:40.693984 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.693966    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-fb4a1b-serving-cert\""
Oct 30 19:19:40.744022 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.743992    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"]
Oct 30 19:19:40.754644 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.754616    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbzt8\" (UniqueName: \"kubernetes.io/projected/79d9bcef-ed50-4e58-9a38-ab11f215ada6-kube-api-access-zbzt8\") pod \"maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:40.754791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.754680    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-metrics-certs\") pod \"maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:40.754867 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.754855    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:40.754938 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.754920    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79d9bcef-ed50-4e58-9a38-ab11f215ada6-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:40.856495 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.856467    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:40.856664 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.856522    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79d9bcef-ed50-4e58-9a38-ab11f215ada6-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:40.856664 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:19:40.856606    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-serving-cert: secret "maas-api-e2e-ipp-a-fb4a1b-serving-cert" not found
Oct 30 19:19:40.856664 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.856648    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbzt8\" (UniqueName: \"kubernetes.io/projected/79d9bcef-ed50-4e58-9a38-ab11f215ada6-kube-api-access-zbzt8\") pod \"maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:40.856848 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:19:40.856680    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-maas-api-tls podName:79d9bcef-ed50-4e58-9a38-ab11f215ada6 nodeName:}" failed. No retries permitted until 2026-10-30 19:19:41.356659389 +0000 UTC m=+1954.302827287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-maas-api-tls") pod "maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj" (UID: "79d9bcef-ed50-4e58-9a38-ab11f215ada6") : secret "maas-api-e2e-ipp-a-fb4a1b-serving-cert" not found
Oct 30 19:19:40.856848 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.856732    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-metrics-certs\") pod \"maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:40.857304 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.857280    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79d9bcef-ed50-4e58-9a38-ab11f215ada6-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:40.859069 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.859053    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-metrics-certs\") pod \"maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:40.862574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.862550    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:40.924862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:40.924787    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbzt8\" (UniqueName: \"kubernetes.io/projected/79d9bcef-ed50-4e58-9a38-ab11f215ada6-kube-api-access-zbzt8\") pod \"maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:41.064648 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:41.064608    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:41.247008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:41.246976    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:41.369553 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:41.369524    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:41.372047 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:41.372010    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-maas-api-tls\") pod \"maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:41.557326 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:41.557245    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:41.929425 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:41.929400    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"]
Oct 30 19:19:41.931403 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:19:41.931374    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79d9bcef_ed50_4e58_9a38_ab11f215ada6.slice/crio-ad55222f15496587060562cf9766202d71dada89bdf1f97c867d23f6d24b4116 WatchSource:0}: Error finding container ad55222f15496587060562cf9766202d71dada89bdf1f97c867d23f6d24b4116: Status 404 returned error can't find the container with id ad55222f15496587060562cf9766202d71dada89bdf1f97c867d23f6d24b4116
Oct 30 19:19:42.644347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:42.644307    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj" event={"ID":"79d9bcef-ed50-4e58-9a38-ab11f215ada6","Type":"ContainerStarted","Data":"ad55222f15496587060562cf9766202d71dada89bdf1f97c867d23f6d24b4116"}
Oct 30 19:19:43.672185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:43.668669    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj" event={"ID":"79d9bcef-ed50-4e58-9a38-ab11f215ada6","Type":"ContainerStarted","Data":"119463e8fdf44c842b43b9466d71458e5c3ced5428b08d265ecd7e7272d79257"}
Oct 30 19:19:43.673684 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:43.673604    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:43.785909 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:43.785846    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj" podStartSLOduration=2.9710708869999998 podStartE2EDuration="3.785829714s" totalImagesPullingTime="814.758827ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:43.784544133 +0000 UTC m=+1956.730712037" watchObservedRunningTime="2026-10-30 19:19:43.785829714 +0000 UTC m=+1956.731997619"
Oct 30 19:19:45.805054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:45.801248    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:50.466461 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:50.466423    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:50.691371 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:50.691341    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:19:50.767161 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:50.767132    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:51.826888 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:51.826855    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:55.217858 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:55.215544    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:58.002608 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:58.002566    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:58.185729 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:58.185698    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:58.394168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:58.394074    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"]
Oct 30 19:19:58.394335 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:58.394288    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf" podUID="3d980aa7-cbc1-4e29-b132-0245bfddd57b" containerName="istio-proxy" containerID="cri-o://4fa705dc7a2071416f602dce5501325d9be8cbf9337a0b23d0adee2d92c67114" gracePeriod=30
Oct 30 19:19:58.439803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:58.439761    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"]
Oct 30 19:19:58.440111 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:58.440083    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r" podUID="66bc2818-931a-410d-a3ad-50ad679764c4" containerName="istio-proxy" containerID="cri-o://86fc574629fa3addf4cfc29b9290b664a83241c61ec1efed8b7bf8f25ed8e4ad" gracePeriod=30
Oct 30 19:19:58.442500 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:58.442478    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"]
Oct 30 19:19:58.442728 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:58.442707    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk" podUID="dbb431a4-89ed-438b-9f24-5573b880f19c" containerName="istio-proxy" containerID="cri-o://be1e80316a6db2509d19a6586ca4b2154be59234fe75094b385d995cbd394dc7" gracePeriod=30
Oct 30 19:19:58.450116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:58.450092    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:58.563802 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:58.563765    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:19:58.564160 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:58.564124    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf" podUID="3d980aa7-cbc1-4e29-b132-0245bfddd57b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:19:58.841009 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:58.840974    5704 patch_prober.go:29] interesting pod/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:19:58.841200 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:58.841028    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r" podUID="66bc2818-931a-410d-a3ad-50ad679764c4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:19:59.244367 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.244275    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:59.659168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.659128    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:19:59.767444 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.767411    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"]
Oct 30 19:19:59.771487 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.771459    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.825160 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.825133    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-324ebe-openshift-default-dockercfg-wgfx5\""
Oct 30 19:19:59.836337 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.836311    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"]
Oct 30 19:19:59.882614 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.882573    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-token\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.882811 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.882638    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istiod-ca-cert\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.882811 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.882735    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-data\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.882811 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.882760    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-socket\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.882918 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.882846    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-envoy\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.882918 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.882896    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpghj\" (UniqueName: \"kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-kube-api-access-rpghj\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.883111 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.883079    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-certs\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.883174 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.883160    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-credential-socket\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.883262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.883239    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-podinfo\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.890524 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.890489    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:19:59.992112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.992002    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpghj\" (UniqueName: \"kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-kube-api-access-rpghj\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.992422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.992399    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-certs\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.992562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.992545    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-credential-socket\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.992684 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.992671    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-podinfo\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.992848 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.992833    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-token\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.996065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.993069    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istiod-ca-cert\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.996317 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.996296    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-data\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.996466 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.996448    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-socket\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:19:59.996638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.996624    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-envoy\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:00.005062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.997604    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-credential-socket\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:00.005363 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.999654    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-data\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:00.005485 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:19:59.999873    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-socket\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:00.005576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.000110    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-certs\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:00.005670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.000725    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istiod-ca-cert\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:00.027640 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.027602    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-envoy\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:00.028470 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.028446    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-podinfo\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:00.052910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.052845    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpghj\" (UniqueName: \"kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-kube-api-access-rpghj\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:00.062192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.060345    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-token\") pod \"e2e-api-a-324ebe-openshift-default-648668947f-s7fkh\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") " pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:00.075057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.072934    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:00.090079 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.088345    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:00.246931 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.246900    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:00.587131 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.587081    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:00.587273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.587182    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:00.587273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.587230    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:00.588397 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.588368    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"]
Oct 30 19:20:00.782065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.778871    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" event={"ID":"096b94a8-ca2d-419e-9ce4-3554b762cbc8","Type":"ContainerStarted","Data":"ed9bcd6f85b66884125389f3c3ba006b2ca67c240a175ef000383d338498e017"}
Oct 30 19:20:00.782065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.778921    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" event={"ID":"096b94a8-ca2d-419e-9ce4-3554b762cbc8","Type":"ContainerStarted","Data":"9ec0bfbb5148993491d73fed3cfacd7f51860e1631632b6dc9ceca0271389b5d"}
Oct 30 19:20:00.898339 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:00.898241    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" podStartSLOduration=1.898225769 podStartE2EDuration="1.898225769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:00.896768381 +0000 UTC m=+1973.842936307" watchObservedRunningTime="2026-10-30 19:20:00.898225769 +0000 UTC m=+1973.844393671"
Oct 30 19:20:01.093194 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:01.093157    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:01.097512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:01.097485    5704 patch_prober.go:29] interesting pod/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 19:20:01.097723 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:01.097704    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 19:20:02.094584 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:02.094551    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:02.791092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:02.791066    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:02.791846 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:02.791830    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:20:03.384615 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:20:03.384567    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:20:03.814687 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.814639    5704 generic.go:356] "Generic (PLEG): container finished" podID="3d980aa7-cbc1-4e29-b132-0245bfddd57b" containerID="4fa705dc7a2071416f602dce5501325d9be8cbf9337a0b23d0adee2d92c67114" exitCode=0
Oct 30 19:20:03.815068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.815003    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf" event={"ID":"3d980aa7-cbc1-4e29-b132-0245bfddd57b","Type":"ContainerDied","Data":"4fa705dc7a2071416f602dce5501325d9be8cbf9337a0b23d0adee2d92c67114"}
Oct 30 19:20:03.815311 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.815260    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf" event={"ID":"3d980aa7-cbc1-4e29-b132-0245bfddd57b","Type":"ContainerDied","Data":"5735d9d9907110ae7727c4ecfa100e16f98447b596539c10747b911f7e33cde2"}
Oct 30 19:20:03.815620 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.815582    5704 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5735d9d9907110ae7727c4ecfa100e16f98447b596539c10747b911f7e33cde2"
Oct 30 19:20:03.817426 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.817355    5704 generic.go:356] "Generic (PLEG): container finished" podID="dbb431a4-89ed-438b-9f24-5573b880f19c" containerID="be1e80316a6db2509d19a6586ca4b2154be59234fe75094b385d995cbd394dc7" exitCode=0
Oct 30 19:20:03.817652 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.817590    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk" event={"ID":"dbb431a4-89ed-438b-9f24-5573b880f19c","Type":"ContainerDied","Data":"be1e80316a6db2509d19a6586ca4b2154be59234fe75094b385d995cbd394dc7"}
Oct 30 19:20:03.853264 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.852900    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:20:03.853750 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.853622    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:03.854383 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.854357    5704 generic.go:356] "Generic (PLEG): container finished" podID="66bc2818-931a-410d-a3ad-50ad679764c4" containerID="86fc574629fa3addf4cfc29b9290b664a83241c61ec1efed8b7bf8f25ed8e4ad" exitCode=0
Oct 30 19:20:03.855462 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.855439    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r" event={"ID":"66bc2818-931a-410d-a3ad-50ad679764c4","Type":"ContainerDied","Data":"86fc574629fa3addf4cfc29b9290b664a83241c61ec1efed8b7bf8f25ed8e4ad"}
Oct 30 19:20:03.878211 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878188    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-certs\") pod \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") "
Oct 30 19:20:03.878317 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878232    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-data\") pod \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") "
Oct 30 19:20:03.878317 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878263    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-credential-socket\") pod \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") "
Oct 30 19:20:03.878398 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878372    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-token\" (UniqueName: \"kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-token\") pod \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") "
Oct 30 19:20:03.878443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878415    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-socket\") pod \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") "
Oct 30 19:20:03.878500 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878456    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-certs" pod "3d980aa7-cbc1-4e29-b132-0245bfddd57b" (UID: "3d980aa7-cbc1-4e29-b132-0245bfddd57b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.878563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878479    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-data" pod "3d980aa7-cbc1-4e29-b132-0245bfddd57b" (UID: "3d980aa7-cbc1-4e29-b132-0245bfddd57b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.878563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878501    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-envoy\") pod \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") "
Oct 30 19:20:03.878563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878521    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-credential-socket" pod "3d980aa7-cbc1-4e29-b132-0245bfddd57b" (UID: "3d980aa7-cbc1-4e29-b132-0245bfddd57b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.878563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878529    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istiod-ca-cert\") pod \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") "
Oct 30 19:20:03.878760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878654    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-socket" pod "3d980aa7-cbc1-4e29-b132-0245bfddd57b" (UID: "3d980aa7-cbc1-4e29-b132-0245bfddd57b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.878760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878668    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-kube-api-access-62sc2\" (UniqueName: \"kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-kube-api-access-62sc2\") pod \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") "
Oct 30 19:20:03.878760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878745    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-podinfo\") pod \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\" (UID: \"3d980aa7-cbc1-4e29-b132-0245bfddd57b\") "
Oct 30 19:20:03.878915 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.878819    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istiod-ca-cert" pod "3d980aa7-cbc1-4e29-b132-0245bfddd57b" (UID: "3d980aa7-cbc1-4e29-b132-0245bfddd57b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:03.879507 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.879457    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.879507 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.879497    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.879634 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.879512    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.879634 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.879525    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.879634 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.879538    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.880624 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.880593    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-kube-api-access-62sc2" pod "3d980aa7-cbc1-4e29-b132-0245bfddd57b" (UID: "3d980aa7-cbc1-4e29-b132-0245bfddd57b"). InnerVolumeSpecName "kube-api-access-62sc2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:03.880833 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.880812    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-envoy" pod "3d980aa7-cbc1-4e29-b132-0245bfddd57b" (UID: "3d980aa7-cbc1-4e29-b132-0245bfddd57b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.881019 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.880996    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-token" pod "3d980aa7-cbc1-4e29-b132-0245bfddd57b" (UID: "3d980aa7-cbc1-4e29-b132-0245bfddd57b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:03.881861 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.881836    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-podinfo" pod "3d980aa7-cbc1-4e29-b132-0245bfddd57b" (UID: "3d980aa7-cbc1-4e29-b132-0245bfddd57b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:20:03.895693 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.895674    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:20:03.898807 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.898790    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:20:03.980213 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980179    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-socket\") pod \"66bc2818-931a-410d-a3ad-50ad679764c4\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") "
Oct 30 19:20:03.980401 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980241    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-envoy\") pod \"dbb431a4-89ed-438b-9f24-5573b880f19c\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") "
Oct 30 19:20:03.980401 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980269    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-socket\") pod \"dbb431a4-89ed-438b-9f24-5573b880f19c\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") "
Oct 30 19:20:03.980401 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980311    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-kube-api-access-wd9zz\" (UniqueName: \"kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-kube-api-access-wd9zz\") pod \"66bc2818-931a-410d-a3ad-50ad679764c4\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") "
Oct 30 19:20:03.980401 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980332    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-istio-token\" (UniqueName: \"kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-istio-token\") pod \"66bc2818-931a-410d-a3ad-50ad679764c4\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") "
Oct 30 19:20:03.980401 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980363    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-data\") pod \"66bc2818-931a-410d-a3ad-50ad679764c4\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") "
Oct 30 19:20:03.980401 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980389    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-credential-socket\") pod \"dbb431a4-89ed-438b-9f24-5573b880f19c\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") "
Oct 30 19:20:03.980709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980424    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-data\") pod \"dbb431a4-89ed-438b-9f24-5573b880f19c\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") "
Oct 30 19:20:03.980709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980457    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/66bc2818-931a-410d-a3ad-50ad679764c4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66bc2818-931a-410d-a3ad-50ad679764c4-istio-podinfo\") pod \"66bc2818-931a-410d-a3ad-50ad679764c4\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") "
Oct 30 19:20:03.980709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980495    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/66bc2818-931a-410d-a3ad-50ad679764c4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66bc2818-931a-410d-a3ad-50ad679764c4-istiod-ca-cert\") pod \"66bc2818-931a-410d-a3ad-50ad679764c4\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") "
Oct 30 19:20:03.980709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980487    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-socket" pod "66bc2818-931a-410d-a3ad-50ad679764c4" (UID: "66bc2818-931a-410d-a3ad-50ad679764c4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.980709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980522    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-credential-socket\") pod \"66bc2818-931a-410d-a3ad-50ad679764c4\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") "
Oct 30 19:20:03.980709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980589    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-certs\") pod \"66bc2818-931a-410d-a3ad-50ad679764c4\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") "
Oct 30 19:20:03.980709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980638    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-envoy\") pod \"66bc2818-931a-410d-a3ad-50ad679764c4\" (UID: \"66bc2818-931a-410d-a3ad-50ad679764c4\") "
Oct 30 19:20:03.980709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980677    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/dbb431a4-89ed-438b-9f24-5573b880f19c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbb431a4-89ed-438b-9f24-5573b880f19c-istiod-ca-cert\") pod \"dbb431a4-89ed-438b-9f24-5573b880f19c\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") "
Oct 30 19:20:03.980709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980700    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-kube-api-access-thpjq\" (UniqueName: \"kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-kube-api-access-thpjq\") pod \"dbb431a4-89ed-438b-9f24-5573b880f19c\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") "
Oct 30 19:20:03.981248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980712    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-socket" pod "dbb431a4-89ed-438b-9f24-5573b880f19c" (UID: "dbb431a4-89ed-438b-9f24-5573b880f19c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.981248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980760    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-podinfo\") pod \"dbb431a4-89ed-438b-9f24-5573b880f19c\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") "
Oct 30 19:20:03.981248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980787    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-credential-socket" pod "dbb431a4-89ed-438b-9f24-5573b880f19c" (UID: "dbb431a4-89ed-438b-9f24-5573b880f19c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.981248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980810    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-certs\") pod \"dbb431a4-89ed-438b-9f24-5573b880f19c\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") "
Oct 30 19:20:03.981248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980836    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-token\" (UniqueName: \"kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-token\") pod \"dbb431a4-89ed-438b-9f24-5573b880f19c\" (UID: \"dbb431a4-89ed-438b-9f24-5573b880f19c\") "
Oct 30 19:20:03.981248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.980917    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-credential-socket" pod "66bc2818-931a-410d-a3ad-50ad679764c4" (UID: "66bc2818-931a-410d-a3ad-50ad679764c4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.981248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.981141    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-data" pod "dbb431a4-89ed-438b-9f24-5573b880f19c" (UID: "dbb431a4-89ed-438b-9f24-5573b880f19c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.981690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.981522    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.981690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.981541    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.981690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.981555    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.981690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.981543    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-data" pod "66bc2818-931a-410d-a3ad-50ad679764c4" (UID: "66bc2818-931a-410d-a3ad-50ad679764c4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.981690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.981568    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.981690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.981587    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.981690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.981601    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.981690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.981615    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62sc2\" (UniqueName: \"kubernetes.io/projected/3d980aa7-cbc1-4e29-b132-0245bfddd57b-kube-api-access-62sc2\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.981690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.981630    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d980aa7-cbc1-4e29-b132-0245bfddd57b-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.981690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.981644    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:03.982411 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.982381    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbb431a4-89ed-438b-9f24-5573b880f19c-istiod-ca-cert" pod "dbb431a4-89ed-438b-9f24-5573b880f19c" (UID: "dbb431a4-89ed-438b-9f24-5573b880f19c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:03.982672 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.982646    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-certs" pod "66bc2818-931a-410d-a3ad-50ad679764c4" (UID: "66bc2818-931a-410d-a3ad-50ad679764c4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.982788 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.982765    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-certs" pod "dbb431a4-89ed-438b-9f24-5573b880f19c" (UID: "dbb431a4-89ed-438b-9f24-5573b880f19c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.982887 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.982860    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66bc2818-931a-410d-a3ad-50ad679764c4-istiod-ca-cert" pod "66bc2818-931a-410d-a3ad-50ad679764c4" (UID: "66bc2818-931a-410d-a3ad-50ad679764c4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:03.982960 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.982911    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-envoy" pod "dbb431a4-89ed-438b-9f24-5573b880f19c" (UID: "dbb431a4-89ed-438b-9f24-5573b880f19c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.983964 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.983935    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-istio-token" pod "66bc2818-931a-410d-a3ad-50ad679764c4" (UID: "66bc2818-931a-410d-a3ad-50ad679764c4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:03.984190 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.984164    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-kube-api-access-wd9zz" pod "66bc2818-931a-410d-a3ad-50ad679764c4" (UID: "66bc2818-931a-410d-a3ad-50ad679764c4"). InnerVolumeSpecName "kube-api-access-wd9zz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:03.984319 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.984298    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-kube-api-access-thpjq" pod "dbb431a4-89ed-438b-9f24-5573b880f19c" (UID: "dbb431a4-89ed-438b-9f24-5573b880f19c"). InnerVolumeSpecName "kube-api-access-thpjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:03.984793 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.984769    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/66bc2818-931a-410d-a3ad-50ad679764c4-istio-podinfo" pod "66bc2818-931a-410d-a3ad-50ad679764c4" (UID: "66bc2818-931a-410d-a3ad-50ad679764c4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:20:03.984881 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.984846    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-envoy" pod "66bc2818-931a-410d-a3ad-50ad679764c4" (UID: "66bc2818-931a-410d-a3ad-50ad679764c4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:20:03.985719 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.985693    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-podinfo" pod "dbb431a4-89ed-438b-9f24-5573b880f19c" (UID: "dbb431a4-89ed-438b-9f24-5573b880f19c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:20:03.985812 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:03.985758    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-token" pod "dbb431a4-89ed-438b-9f24-5573b880f19c" (UID: "dbb431a4-89ed-438b-9f24-5573b880f19c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:04.074586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.074495    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:04.082673 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.082651    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.082816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.082676    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.082816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.082692    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.082816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.082706    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd9zz\" (UniqueName: \"kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-kube-api-access-wd9zz\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.082816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.082719    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/66bc2818-931a-410d-a3ad-50ad679764c4-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.082816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.082732    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.082816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.082748    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/66bc2818-931a-410d-a3ad-50ad679764c4-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.082816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.082760    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/66bc2818-931a-410d-a3ad-50ad679764c4-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.082816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.082773    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.082816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.082784    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/66bc2818-931a-410d-a3ad-50ad679764c4-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.082816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.082797    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dbb431a4-89ed-438b-9f24-5573b880f19c-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.082816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.082810    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thpjq\" (UniqueName: \"kubernetes.io/projected/dbb431a4-89ed-438b-9f24-5573b880f19c-kube-api-access-thpjq\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.083206 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.082823    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dbb431a4-89ed-438b-9f24-5573b880f19c-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:04.863245 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.863217    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"
Oct 30 19:20:04.863245 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.863236    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk" event={"ID":"dbb431a4-89ed-438b-9f24-5573b880f19c","Type":"ContainerDied","Data":"2a630408f894d26cb431473ae6221aab71560db2f5691127cf577f38d94faf53"}
Oct 30 19:20:04.863780 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.863300    5704 scope.go:122] "RemoveContainer" containerID="be1e80316a6db2509d19a6586ca4b2154be59234fe75094b385d995cbd394dc7"
Oct 30 19:20:04.865158 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.865079    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r" event={"ID":"66bc2818-931a-410d-a3ad-50ad679764c4","Type":"ContainerDied","Data":"8621cd897809e228859cfb958b33494c647ce2631487d9b92d972737044522ba"}
Oct 30 19:20:04.865158 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.865135    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"
Oct 30 19:20:04.865329 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.865296    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"
Oct 30 19:20:04.876064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.876011    5704 scope.go:122] "RemoveContainer" containerID="86fc574629fa3addf4cfc29b9290b664a83241c61ec1efed8b7bf8f25ed8e4ad"
Oct 30 19:20:04.927056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:04.917795    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:05.057895 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:05.057850    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"]
Oct 30 19:20:05.058813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:05.058786    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-svhzk"]
Oct 30 19:20:05.147085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:05.146987    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"]
Oct 30 19:20:05.149505 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:05.149479    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-4ht2r"]
Oct 30 19:20:05.180500 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:05.180477    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66bc2818-931a-410d-a3ad-50ad679764c4" path="/var/lib/kubelet/pods/66bc2818-931a-410d-a3ad-50ad679764c4/volumes"
Oct 30 19:20:05.180975 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:05.180960    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dbb431a4-89ed-438b-9f24-5573b880f19c" path="/var/lib/kubelet/pods/dbb431a4-89ed-438b-9f24-5573b880f19c/volumes"
Oct 30 19:20:05.248148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:05.248114    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"]
Oct 30 19:20:05.252260 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:05.252236    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-942c5e85-openshift-default-845cb4548f-sm7hf"]
Oct 30 19:20:05.504406 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:05.504377    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:06.993125 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:06.993089    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:07.178070 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.177807    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:07.184268 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.184236    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d980aa7-cbc1-4e29-b132-0245bfddd57b" path="/var/lib/kubelet/pods/3d980aa7-cbc1-4e29-b132-0245bfddd57b/volumes"
Oct 30 19:20:07.576021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.571589    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"]
Oct 30 19:20:07.576021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.572559    5704 cpu_manager.go:414] "containerMap: removing container" podUID="3d980aa7-cbc1-4e29-b132-0245bfddd57b" containerName="istio-proxy"
Oct 30 19:20:07.576021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.572583    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d980aa7-cbc1-4e29-b132-0245bfddd57b" containerName="istio-proxy"
Oct 30 19:20:07.576021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.572607    5704 cpu_manager.go:414] "containerMap: removing container" podUID="dbb431a4-89ed-438b-9f24-5573b880f19c" containerName="istio-proxy"
Oct 30 19:20:07.576021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.572615    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dbb431a4-89ed-438b-9f24-5573b880f19c" containerName="istio-proxy"
Oct 30 19:20:07.576021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.572661    5704 cpu_manager.go:414] "containerMap: removing container" podUID="66bc2818-931a-410d-a3ad-50ad679764c4" containerName="istio-proxy"
Oct 30 19:20:07.576021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.572671    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66bc2818-931a-410d-a3ad-50ad679764c4" containerName="istio-proxy"
Oct 30 19:20:07.576021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.572849    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d980aa7-cbc1-4e29-b132-0245bfddd57b" containerName="istio-proxy"
Oct 30 19:20:07.576021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.572870    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="dbb431a4-89ed-438b-9f24-5573b880f19c" containerName="istio-proxy"
Oct 30 19:20:07.576021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.572889    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="66bc2818-931a-410d-a3ad-50ad679764c4" containerName="istio-proxy"
Oct 30 19:20:07.620200 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.620171    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"]
Oct 30 19:20:07.620502 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.620488    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"
Oct 30 19:20:07.657488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.657252    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae1156ad-72a2-460f-95f5-bb4c7048ed94-plugins-config-volume\") pod \"payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb\" (UID: \"ae1156ad-72a2-460f-95f5-bb4c7048ed94\") " pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"
Oct 30 19:20:07.657488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.657316    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq5hv\" (UniqueName: \"kubernetes.io/projected/ae1156ad-72a2-460f-95f5-bb4c7048ed94-kube-api-access-dq5hv\") pod \"payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb\" (UID: \"ae1156ad-72a2-460f-95f5-bb4c7048ed94\") " pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"
Oct 30 19:20:07.715979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.715954    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-324ebe-dockercfg-59xxk\""
Oct 30 19:20:07.715979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.715954    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-324ebe\""
Oct 30 19:20:07.758674 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.758642    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae1156ad-72a2-460f-95f5-bb4c7048ed94-plugins-config-volume\") pod \"payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb\" (UID: \"ae1156ad-72a2-460f-95f5-bb4c7048ed94\") " pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"
Oct 30 19:20:07.758889 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.758693    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq5hv\" (UniqueName: \"kubernetes.io/projected/ae1156ad-72a2-460f-95f5-bb4c7048ed94-kube-api-access-dq5hv\") pod \"payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb\" (UID: \"ae1156ad-72a2-460f-95f5-bb4c7048ed94\") " pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"
Oct 30 19:20:07.759453 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.759429    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae1156ad-72a2-460f-95f5-bb4c7048ed94-plugins-config-volume\") pod \"payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb\" (UID: \"ae1156ad-72a2-460f-95f5-bb4c7048ed94\") " pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"
Oct 30 19:20:07.817212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.817181    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq5hv\" (UniqueName: \"kubernetes.io/projected/ae1156ad-72a2-460f-95f5-bb4c7048ed94-kube-api-access-dq5hv\") pod \"payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb\" (UID: \"ae1156ad-72a2-460f-95f5-bb4c7048ed94\") " pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"
Oct 30 19:20:07.952882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:07.952793    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"
Oct 30 19:20:08.329122 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:08.329095    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"]
Oct 30 19:20:08.331321 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:20:08.331295    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae1156ad_72a2_460f_95f5_bb4c7048ed94.slice/crio-b33002a5ddb91f68e578c1537f75bb695ece4b94b9cfd12c61e0af5d16ae3c28 WatchSource:0}: Error finding container b33002a5ddb91f68e578c1537f75bb695ece4b94b9cfd12c61e0af5d16ae3c28: Status 404 returned error can't find the container with id b33002a5ddb91f68e578c1537f75bb695ece4b94b9cfd12c61e0af5d16ae3c28
Oct 30 19:20:08.897402 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:08.897366    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb" event={"ID":"ae1156ad-72a2-460f-95f5-bb4c7048ed94","Type":"ContainerStarted","Data":"1152c73a0aa03b7eb8aa0e4f9d4b2e04d2197d17f91d87053831e80dbb1f7000"}
Oct 30 19:20:08.897567 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:08.897415    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb" event={"ID":"ae1156ad-72a2-460f-95f5-bb4c7048ed94","Type":"ContainerStarted","Data":"b33002a5ddb91f68e578c1537f75bb695ece4b94b9cfd12c61e0af5d16ae3c28"}
Oct 30 19:20:08.897567 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:08.897503    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"
Oct 30 19:20:09.019069 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:09.018089    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb" podStartSLOduration=2.018070519 podStartE2EDuration="2.018070519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:09.015264748 +0000 UTC m=+1981.961432652" watchObservedRunningTime="2026-10-30 19:20:09.018070519 +0000 UTC m=+1981.964238423"
Oct 30 19:20:09.721093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:09.713305    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:10.911841 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:10.911805    5704 generic.go:356] "Generic (PLEG): container finished" podID="793048dc-32db-41ad-9964-04c275d02cbf" containerID="8dfe37d6425983618fd225a98a8621120318032baaccc38644649cee998d5ecb" exitCode=0
Oct 30 19:20:10.912326 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:10.911878    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf" event={"ID":"793048dc-32db-41ad-9964-04c275d02cbf","Type":"ContainerDied","Data":"8dfe37d6425983618fd225a98a8621120318032baaccc38644649cee998d5ecb"}
Oct 30 19:20:10.912326 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:10.912319    5704 scope.go:122] "RemoveContainer" containerID="8dfe37d6425983618fd225a98a8621120318032baaccc38644649cee998d5ecb"
Oct 30 19:20:10.913921 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:10.913891    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:10.913983 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:10.913964    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:10.914020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:10.913996    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:20:11.669874 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:11.669832    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:11.946317 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:11.946232    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf" event={"ID":"793048dc-32db-41ad-9964-04c275d02cbf","Type":"ContainerStarted","Data":"3072ecd263b3d61aec49aebf52648071def1d44f424ba46ed9795ef2af7b4727"}
Oct 30 19:20:12.494556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:12.494528    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:20:12.494741 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:12.494661    5704 patch_prober.go:29] interesting pod/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:20:12.494741 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:12.494714    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf" podUID="793048dc-32db-41ad-9964-04c275d02cbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 19:20:12.603591 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:12.603554    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:13.494956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:13.494922    5704 patch_prober.go:29] interesting pod/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:20:13.495397 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:13.494982    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf" podUID="793048dc-32db-41ad-9964-04c275d02cbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 19:20:14.495325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:14.495295    5704 patch_prober.go:29] interesting pod/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:20:14.495762 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:14.495344    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf" podUID="793048dc-32db-41ad-9964-04c275d02cbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 19:20:14.741061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:14.732698    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:15.046596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:15.046558    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"]
Oct 30 19:20:15.146884 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:15.146843    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"]
Oct 30 19:20:15.340109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:15.339184    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:15.495371 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:15.495338    5704 patch_prober.go:29] interesting pod/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:20:15.495750 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:15.495395    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf" podUID="793048dc-32db-41ad-9964-04c275d02cbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 19:20:15.515315 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:15.515286    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:15.697684 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:15.697607    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:15.982530 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:15.982452    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:16.498334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:16.498299    5704 patch_prober.go:29] interesting pod/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused" start-of-body=
Oct 30 19:20:16.498905 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:16.498871    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf" podUID="793048dc-32db-41ad-9964-04c275d02cbf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.96:15021/healthz/ready\": dial tcp 10.134.0.96:15021: connect: connection refused"
Oct 30 19:20:17.499176 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:17.499147    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:20:17.499558 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:17.499518    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:20:17.500088 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:17.500063    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:20:19.767353 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:19.767321    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:19.907940 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:19.907911    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"
Oct 30 19:20:20.047551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.047467    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf" podUID="8db7e37a-9924-4ce9-bac0-67edd0374bf0" containerName="payload-processing" containerID="cri-o://6f20d74cb970cbc0e05528997639e29d6516a60326647a6f3fd2684841eddce8" gracePeriod=25
Oct 30 19:20:20.147134 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.147095    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb" podUID="eb1c92a8-55e3-46fd-ae18-410645416414" containerName="payload-pre-processing" containerID="cri-o://7e7499637c0cdbda4f68f172120da8b49080f092289e59632692ffec45d67ab6" gracePeriod=25
Oct 30 19:20:20.317352 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.317327    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"
Oct 30 19:20:20.370470 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.370442    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8db7e37a-9924-4ce9-bac0-67edd0374bf0-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db7e37a-9924-4ce9-bac0-67edd0374bf0-plugins-config-volume\") pod \"8db7e37a-9924-4ce9-bac0-67edd0374bf0\" (UID: \"8db7e37a-9924-4ce9-bac0-67edd0374bf0\") "
Oct 30 19:20:20.370651 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.370540    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8db7e37a-9924-4ce9-bac0-67edd0374bf0-kube-api-access-mq5nb\" (UniqueName: \"kubernetes.io/projected/8db7e37a-9924-4ce9-bac0-67edd0374bf0-kube-api-access-mq5nb\") pod \"8db7e37a-9924-4ce9-bac0-67edd0374bf0\" (UID: \"8db7e37a-9924-4ce9-bac0-67edd0374bf0\") "
Oct 30 19:20:20.370916 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.370894    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8db7e37a-9924-4ce9-bac0-67edd0374bf0-plugins-config-volume" pod "8db7e37a-9924-4ce9-bac0-67edd0374bf0" (UID: "8db7e37a-9924-4ce9-bac0-67edd0374bf0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:20.371288 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.371268    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8db7e37a-9924-4ce9-bac0-67edd0374bf0-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:20.372657 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.372628    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db7e37a-9924-4ce9-bac0-67edd0374bf0-kube-api-access-mq5nb" pod "8db7e37a-9924-4ce9-bac0-67edd0374bf0" (UID: "8db7e37a-9924-4ce9-bac0-67edd0374bf0"). InnerVolumeSpecName "kube-api-access-mq5nb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:20.392983 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.392957    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"
Oct 30 19:20:20.472511 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.472472    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb1c92a8-55e3-46fd-ae18-410645416414-kube-api-access-crdw2\" (UniqueName: \"kubernetes.io/projected/eb1c92a8-55e3-46fd-ae18-410645416414-kube-api-access-crdw2\") pod \"eb1c92a8-55e3-46fd-ae18-410645416414\" (UID: \"eb1c92a8-55e3-46fd-ae18-410645416414\") "
Oct 30 19:20:20.472664 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.472623    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb1c92a8-55e3-46fd-ae18-410645416414-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1c92a8-55e3-46fd-ae18-410645416414-plugins-config-volume\") pod \"eb1c92a8-55e3-46fd-ae18-410645416414\" (UID: \"eb1c92a8-55e3-46fd-ae18-410645416414\") "
Oct 30 19:20:20.473083 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.473053    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb1c92a8-55e3-46fd-ae18-410645416414-plugins-config-volume" pod "eb1c92a8-55e3-46fd-ae18-410645416414" (UID: "eb1c92a8-55e3-46fd-ae18-410645416414"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:20.473474 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.473451    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mq5nb\" (UniqueName: \"kubernetes.io/projected/8db7e37a-9924-4ce9-bac0-67edd0374bf0-kube-api-access-mq5nb\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:20.473555 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.473481    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb1c92a8-55e3-46fd-ae18-410645416414-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:20.474614 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.474591    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb1c92a8-55e3-46fd-ae18-410645416414-kube-api-access-crdw2" pod "eb1c92a8-55e3-46fd-ae18-410645416414" (UID: "eb1c92a8-55e3-46fd-ae18-410645416414"). InnerVolumeSpecName "kube-api-access-crdw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:20.574635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:20.574563    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crdw2\" (UniqueName: \"kubernetes.io/projected/eb1c92a8-55e3-46fd-ae18-410645416414-kube-api-access-crdw2\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:21.007382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.007346    5704 generic.go:356] "Generic (PLEG): container finished" podID="8db7e37a-9924-4ce9-bac0-67edd0374bf0" containerID="6f20d74cb970cbc0e05528997639e29d6516a60326647a6f3fd2684841eddce8" exitCode=0
Oct 30 19:20:21.007855 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.007393    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"
Oct 30 19:20:21.007855 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.007419    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf" event={"ID":"8db7e37a-9924-4ce9-bac0-67edd0374bf0","Type":"ContainerDied","Data":"6f20d74cb970cbc0e05528997639e29d6516a60326647a6f3fd2684841eddce8"}
Oct 30 19:20:21.007855 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.007465    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf" event={"ID":"8db7e37a-9924-4ce9-bac0-67edd0374bf0","Type":"ContainerDied","Data":"a9cc2e6ed5db174326664b98cce9a6d20c73d920c097b1d5de689ee3c9bfb304"}
Oct 30 19:20:21.007855 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.007486    5704 scope.go:122] "RemoveContainer" containerID="6f20d74cb970cbc0e05528997639e29d6516a60326647a6f3fd2684841eddce8"
Oct 30 19:20:21.008902 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.008875    5704 generic.go:356] "Generic (PLEG): container finished" podID="eb1c92a8-55e3-46fd-ae18-410645416414" containerID="7e7499637c0cdbda4f68f172120da8b49080f092289e59632692ffec45d67ab6" exitCode=0
Oct 30 19:20:21.009073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.008916    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"
Oct 30 19:20:21.009073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.008947    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb" event={"ID":"eb1c92a8-55e3-46fd-ae18-410645416414","Type":"ContainerDied","Data":"7e7499637c0cdbda4f68f172120da8b49080f092289e59632692ffec45d67ab6"}
Oct 30 19:20:21.009073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.009014    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb" event={"ID":"eb1c92a8-55e3-46fd-ae18-410645416414","Type":"ContainerDied","Data":"ac7b29796e636f0e2001dded5e1145c167edcc59adc4d77a05ee202a039d75c2"}
Oct 30 19:20:21.019020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.019003    5704 scope.go:122] "RemoveContainer" containerID="6f20d74cb970cbc0e05528997639e29d6516a60326647a6f3fd2684841eddce8"
Oct 30 19:20:21.019291 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:20:21.019272    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f20d74cb970cbc0e05528997639e29d6516a60326647a6f3fd2684841eddce8\": container with ID starting with 6f20d74cb970cbc0e05528997639e29d6516a60326647a6f3fd2684841eddce8 not found: ID does not exist" containerID="6f20d74cb970cbc0e05528997639e29d6516a60326647a6f3fd2684841eddce8"
Oct 30 19:20:21.019349 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.019297    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f20d74cb970cbc0e05528997639e29d6516a60326647a6f3fd2684841eddce8"} err="failed to get container status \"6f20d74cb970cbc0e05528997639e29d6516a60326647a6f3fd2684841eddce8\": rpc error: code = NotFound desc = could not find container \"6f20d74cb970cbc0e05528997639e29d6516a60326647a6f3fd2684841eddce8\": container with ID starting with 6f20d74cb970cbc0e05528997639e29d6516a60326647a6f3fd2684841eddce8 not found: ID does not exist"
Oct 30 19:20:21.019349 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.019312    5704 scope.go:122] "RemoveContainer" containerID="7e7499637c0cdbda4f68f172120da8b49080f092289e59632692ffec45d67ab6"
Oct 30 19:20:21.027754 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.027737    5704 scope.go:122] "RemoveContainer" containerID="7e7499637c0cdbda4f68f172120da8b49080f092289e59632692ffec45d67ab6"
Oct 30 19:20:21.028009 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:20:21.027991    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e7499637c0cdbda4f68f172120da8b49080f092289e59632692ffec45d67ab6\": container with ID starting with 7e7499637c0cdbda4f68f172120da8b49080f092289e59632692ffec45d67ab6 not found: ID does not exist" containerID="7e7499637c0cdbda4f68f172120da8b49080f092289e59632692ffec45d67ab6"
Oct 30 19:20:21.028088 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.028019    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e7499637c0cdbda4f68f172120da8b49080f092289e59632692ffec45d67ab6"} err="failed to get container status \"7e7499637c0cdbda4f68f172120da8b49080f092289e59632692ffec45d67ab6\": rpc error: code = NotFound desc = could not find container \"7e7499637c0cdbda4f68f172120da8b49080f092289e59632692ffec45d67ab6\": container with ID starting with 7e7499637c0cdbda4f68f172120da8b49080f092289e59632692ffec45d67ab6 not found: ID does not exist"
Oct 30 19:20:21.160855 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.160829    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"]
Oct 30 19:20:21.166967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.166944    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-54b749bf5-nxhcb"]
Oct 30 19:20:21.179972 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.179952    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb1c92a8-55e3-46fd-ae18-410645416414" path="/var/lib/kubelet/pods/eb1c92a8-55e3-46fd-ae18-410645416414/volumes"
Oct 30 19:20:21.269085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.269004    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"]
Oct 30 19:20:21.274000 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:21.273979    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-905566-67fd5cf94b-vhptf"]
Oct 30 19:20:23.179662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:23.179632    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8db7e37a-9924-4ce9-bac0-67edd0374bf0" path="/var/lib/kubelet/pods/8db7e37a-9924-4ce9-bac0-67edd0374bf0/volumes"
Oct 30 19:20:24.425058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:24.423726    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:29.468460 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:29.468103    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:30.913422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:30.913388    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"]
Oct 30 19:20:31.014423 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:31.014392    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"]
Oct 30 19:20:31.245443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:31.245410    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:31.461054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:31.460897    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:32.187944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:32.187877    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:32.974742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:32.974648    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:33.428129 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:20:33.428091    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:20:33.447180 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:33.447148    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:33.569207 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:33.569175    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"]
Oct 30 19:20:33.666641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:33.666613    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"]
Oct 30 19:20:33.939441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:33.939407    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:34.133577 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:34.133543    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:34.841502 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:34.840906    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:35.914229 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:35.914164    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992" podUID="930f1ac9-f21e-4f2a-ae60-43b41309469b" containerName="payload-processing" containerID="cri-o://d3af5764f86b027af0209d6642903e65b4f802b959c652a58183b4deeca8f047" gracePeriod=25
Oct 30 19:20:36.014658 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.014620    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b" podUID="e29dbede-923d-430c-895a-c28b4c89d0d7" containerName="payload-pre-processing" containerID="cri-o://4d63ef3a72f6e3519440e0e696e31d180758f23b3d030839c67fc83b99fbf102" gracePeriod=25
Oct 30 19:20:36.106515 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.106466    5704 generic.go:356] "Generic (PLEG): container finished" podID="930f1ac9-f21e-4f2a-ae60-43b41309469b" containerID="d3af5764f86b027af0209d6642903e65b4f802b959c652a58183b4deeca8f047" exitCode=0
Oct 30 19:20:36.106681 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.106542    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992" event={"ID":"930f1ac9-f21e-4f2a-ae60-43b41309469b","Type":"ContainerDied","Data":"d3af5764f86b027af0209d6642903e65b4f802b959c652a58183b4deeca8f047"}
Oct 30 19:20:36.187470 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.187407    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"
Oct 30 19:20:36.312488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.312462    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"
Oct 30 19:20:36.321474 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.321439    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/930f1ac9-f21e-4f2a-ae60-43b41309469b-kube-api-access-fln9k\" (UniqueName: \"kubernetes.io/projected/930f1ac9-f21e-4f2a-ae60-43b41309469b-kube-api-access-fln9k\") pod \"930f1ac9-f21e-4f2a-ae60-43b41309469b\" (UID: \"930f1ac9-f21e-4f2a-ae60-43b41309469b\") "
Oct 30 19:20:36.321617 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.321489    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e29dbede-923d-430c-895a-c28b4c89d0d7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e29dbede-923d-430c-895a-c28b4c89d0d7-plugins-config-volume\") pod \"e29dbede-923d-430c-895a-c28b4c89d0d7\" (UID: \"e29dbede-923d-430c-895a-c28b4c89d0d7\") "
Oct 30 19:20:36.321617 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.321541    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e29dbede-923d-430c-895a-c28b4c89d0d7-kube-api-access-flbdf\" (UniqueName: \"kubernetes.io/projected/e29dbede-923d-430c-895a-c28b4c89d0d7-kube-api-access-flbdf\") pod \"e29dbede-923d-430c-895a-c28b4c89d0d7\" (UID: \"e29dbede-923d-430c-895a-c28b4c89d0d7\") "
Oct 30 19:20:36.321738 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.321688    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/930f1ac9-f21e-4f2a-ae60-43b41309469b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/930f1ac9-f21e-4f2a-ae60-43b41309469b-plugins-config-volume\") pod \"930f1ac9-f21e-4f2a-ae60-43b41309469b\" (UID: \"930f1ac9-f21e-4f2a-ae60-43b41309469b\") "
Oct 30 19:20:36.321830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.321803    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e29dbede-923d-430c-895a-c28b4c89d0d7-plugins-config-volume" pod "e29dbede-923d-430c-895a-c28b4c89d0d7" (UID: "e29dbede-923d-430c-895a-c28b4c89d0d7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:36.322071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.322046    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/930f1ac9-f21e-4f2a-ae60-43b41309469b-plugins-config-volume" pod "930f1ac9-f21e-4f2a-ae60-43b41309469b" (UID: "930f1ac9-f21e-4f2a-ae60-43b41309469b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:36.322242 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.322225    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e29dbede-923d-430c-895a-c28b4c89d0d7-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:36.322326 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.322243    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/930f1ac9-f21e-4f2a-ae60-43b41309469b-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:36.323609 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.323586    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e29dbede-923d-430c-895a-c28b4c89d0d7-kube-api-access-flbdf" pod "e29dbede-923d-430c-895a-c28b4c89d0d7" (UID: "e29dbede-923d-430c-895a-c28b4c89d0d7"). InnerVolumeSpecName "kube-api-access-flbdf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:36.323996 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.323975    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/930f1ac9-f21e-4f2a-ae60-43b41309469b-kube-api-access-fln9k" pod "930f1ac9-f21e-4f2a-ae60-43b41309469b" (UID: "930f1ac9-f21e-4f2a-ae60-43b41309469b"). InnerVolumeSpecName "kube-api-access-fln9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:36.423044 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.423008    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fln9k\" (UniqueName: \"kubernetes.io/projected/930f1ac9-f21e-4f2a-ae60-43b41309469b-kube-api-access-fln9k\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:36.423044 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:36.423044    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-flbdf\" (UniqueName: \"kubernetes.io/projected/e29dbede-923d-430c-895a-c28b4c89d0d7-kube-api-access-flbdf\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:37.115121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.115091    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"
Oct 30 19:20:37.115121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.115100    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992" event={"ID":"930f1ac9-f21e-4f2a-ae60-43b41309469b","Type":"ContainerDied","Data":"57f8e3cd52030a1624524282a9384c597db21e4191848aa4e3b234df9277b583"}
Oct 30 19:20:37.115655 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.115159    5704 scope.go:122] "RemoveContainer" containerID="d3af5764f86b027af0209d6642903e65b4f802b959c652a58183b4deeca8f047"
Oct 30 19:20:37.116671 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.116638    5704 generic.go:356] "Generic (PLEG): container finished" podID="e29dbede-923d-430c-895a-c28b4c89d0d7" containerID="4d63ef3a72f6e3519440e0e696e31d180758f23b3d030839c67fc83b99fbf102" exitCode=0
Oct 30 19:20:37.116798 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.116672    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b" event={"ID":"e29dbede-923d-430c-895a-c28b4c89d0d7","Type":"ContainerDied","Data":"4d63ef3a72f6e3519440e0e696e31d180758f23b3d030839c67fc83b99fbf102"}
Oct 30 19:20:37.116798 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.116679    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"
Oct 30 19:20:37.116798 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.116704    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b" event={"ID":"e29dbede-923d-430c-895a-c28b4c89d0d7","Type":"ContainerDied","Data":"827071c1860d03fc381d3ef55941b1c28217b6fafeac70cf1aadb88d3b0fb39f"}
Oct 30 19:20:37.125376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.125358    5704 scope.go:122] "RemoveContainer" containerID="4d63ef3a72f6e3519440e0e696e31d180758f23b3d030839c67fc83b99fbf102"
Oct 30 19:20:37.134000 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.133981    5704 scope.go:122] "RemoveContainer" containerID="4d63ef3a72f6e3519440e0e696e31d180758f23b3d030839c67fc83b99fbf102"
Oct 30 19:20:37.134290 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:20:37.134274    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d63ef3a72f6e3519440e0e696e31d180758f23b3d030839c67fc83b99fbf102\": container with ID starting with 4d63ef3a72f6e3519440e0e696e31d180758f23b3d030839c67fc83b99fbf102 not found: ID does not exist" containerID="4d63ef3a72f6e3519440e0e696e31d180758f23b3d030839c67fc83b99fbf102"
Oct 30 19:20:37.134349 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.134297    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d63ef3a72f6e3519440e0e696e31d180758f23b3d030839c67fc83b99fbf102"} err="failed to get container status \"4d63ef3a72f6e3519440e0e696e31d180758f23b3d030839c67fc83b99fbf102\": rpc error: code = NotFound desc = could not find container \"4d63ef3a72f6e3519440e0e696e31d180758f23b3d030839c67fc83b99fbf102\": container with ID starting with 4d63ef3a72f6e3519440e0e696e31d180758f23b3d030839c67fc83b99fbf102 not found: ID does not exist"
Oct 30 19:20:37.270825 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.270796    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"]
Oct 30 19:20:37.316714 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.316685    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4ec347-69678699bd-b7992"]
Oct 30 19:20:37.379123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.379093    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"]
Oct 30 19:20:37.384621 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.384593    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-59786fb89f-kmc8b"]
Oct 30 19:20:37.631471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.631391    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:37.862569 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:37.862538    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:38.040852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:38.040822    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:38.569637 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:38.569590    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf" podUID="a933f15e-0fa7-4c6d-99a2-b3c4ba0250da" containerName="payload-processing" containerID="cri-o://09c3cbfdcec3d7a9720b58d695b1da75bf8dd9c5dcd034a3636390478203a680" gracePeriod=25
Oct 30 19:20:38.666922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:38.666891    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r" podUID="298df6c6-b7d1-46e2-bbc9-09a000ae3f30" containerName="payload-pre-processing" containerID="cri-o://e7de5fe6ade0ce20b5d647f072cf7a3ea21697cb1b1793cef40e963df2a7f19f" gracePeriod=25
Oct 30 19:20:38.849102 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:38.849028    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:38.923308 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:38.923285    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"
Oct 30 19:20:38.948525 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:38.948497    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-plugins-config-volume\") pod \"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da\" (UID: \"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da\") "
Oct 30 19:20:38.948703 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:38.948686    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-kube-api-access-sv9qc\" (UniqueName: \"kubernetes.io/projected/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-kube-api-access-sv9qc\") pod \"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da\" (UID: \"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da\") "
Oct 30 19:20:38.948914 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:38.948886    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-plugins-config-volume" pod "a933f15e-0fa7-4c6d-99a2-b3c4ba0250da" (UID: "a933f15e-0fa7-4c6d-99a2-b3c4ba0250da"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:38.949281 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:38.949251    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:38.950848 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:38.950825    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-kube-api-access-sv9qc" pod "a933f15e-0fa7-4c6d-99a2-b3c4ba0250da" (UID: "a933f15e-0fa7-4c6d-99a2-b3c4ba0250da"). InnerVolumeSpecName "kube-api-access-sv9qc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:38.968695 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:38.968677    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"
Oct 30 19:20:39.050273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.050245    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-kube-api-access-qddd6\" (UniqueName: \"kubernetes.io/projected/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-kube-api-access-qddd6\") pod \"298df6c6-b7d1-46e2-bbc9-09a000ae3f30\" (UID: \"298df6c6-b7d1-46e2-bbc9-09a000ae3f30\") "
Oct 30 19:20:39.050407 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.050334    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-plugins-config-volume\") pod \"298df6c6-b7d1-46e2-bbc9-09a000ae3f30\" (UID: \"298df6c6-b7d1-46e2-bbc9-09a000ae3f30\") "
Oct 30 19:20:39.050630 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.050605    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-plugins-config-volume" pod "298df6c6-b7d1-46e2-bbc9-09a000ae3f30" (UID: "298df6c6-b7d1-46e2-bbc9-09a000ae3f30"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:39.050954 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.050930    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:39.051092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.050958    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sv9qc\" (UniqueName: \"kubernetes.io/projected/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da-kube-api-access-sv9qc\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:39.052615 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.052587    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-kube-api-access-qddd6" pod "298df6c6-b7d1-46e2-bbc9-09a000ae3f30" (UID: "298df6c6-b7d1-46e2-bbc9-09a000ae3f30"). InnerVolumeSpecName "kube-api-access-qddd6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:39.132466 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.132391    5704 generic.go:356] "Generic (PLEG): container finished" podID="298df6c6-b7d1-46e2-bbc9-09a000ae3f30" containerID="e7de5fe6ade0ce20b5d647f072cf7a3ea21697cb1b1793cef40e963df2a7f19f" exitCode=0
Oct 30 19:20:39.132466 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.132433    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"
Oct 30 19:20:39.132678 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.132473    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r" event={"ID":"298df6c6-b7d1-46e2-bbc9-09a000ae3f30","Type":"ContainerDied","Data":"e7de5fe6ade0ce20b5d647f072cf7a3ea21697cb1b1793cef40e963df2a7f19f"}
Oct 30 19:20:39.132678 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.132539    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r" event={"ID":"298df6c6-b7d1-46e2-bbc9-09a000ae3f30","Type":"ContainerDied","Data":"aea3d384cabdea84887da9383dcec170318533cfd57fc2d964690b582cccf2bc"}
Oct 30 19:20:39.132678 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.132569    5704 scope.go:122] "RemoveContainer" containerID="e7de5fe6ade0ce20b5d647f072cf7a3ea21697cb1b1793cef40e963df2a7f19f"
Oct 30 19:20:39.133983 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.133959    5704 generic.go:356] "Generic (PLEG): container finished" podID="a933f15e-0fa7-4c6d-99a2-b3c4ba0250da" containerID="09c3cbfdcec3d7a9720b58d695b1da75bf8dd9c5dcd034a3636390478203a680" exitCode=0
Oct 30 19:20:39.134134 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.133999    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf" event={"ID":"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da","Type":"ContainerDied","Data":"09c3cbfdcec3d7a9720b58d695b1da75bf8dd9c5dcd034a3636390478203a680"}
Oct 30 19:20:39.134134 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.134009    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"
Oct 30 19:20:39.134134 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.134080    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf" event={"ID":"a933f15e-0fa7-4c6d-99a2-b3c4ba0250da","Type":"ContainerDied","Data":"479618d8a8248d1a2e33957f02ffec691feb33a11d9c8d8a15787d820549f3e6"}
Oct 30 19:20:39.145061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.145022    5704 scope.go:122] "RemoveContainer" containerID="e7de5fe6ade0ce20b5d647f072cf7a3ea21697cb1b1793cef40e963df2a7f19f"
Oct 30 19:20:39.145368 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:20:39.145350    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7de5fe6ade0ce20b5d647f072cf7a3ea21697cb1b1793cef40e963df2a7f19f\": container with ID starting with e7de5fe6ade0ce20b5d647f072cf7a3ea21697cb1b1793cef40e963df2a7f19f not found: ID does not exist" containerID="e7de5fe6ade0ce20b5d647f072cf7a3ea21697cb1b1793cef40e963df2a7f19f"
Oct 30 19:20:39.145441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.145379    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7de5fe6ade0ce20b5d647f072cf7a3ea21697cb1b1793cef40e963df2a7f19f"} err="failed to get container status \"e7de5fe6ade0ce20b5d647f072cf7a3ea21697cb1b1793cef40e963df2a7f19f\": rpc error: code = NotFound desc = could not find container \"e7de5fe6ade0ce20b5d647f072cf7a3ea21697cb1b1793cef40e963df2a7f19f\": container with ID starting with e7de5fe6ade0ce20b5d647f072cf7a3ea21697cb1b1793cef40e963df2a7f19f not found: ID does not exist"
Oct 30 19:20:39.145441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.145401    5704 scope.go:122] "RemoveContainer" containerID="09c3cbfdcec3d7a9720b58d695b1da75bf8dd9c5dcd034a3636390478203a680"
Oct 30 19:20:39.152366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.152343    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qddd6\" (UniqueName: \"kubernetes.io/projected/298df6c6-b7d1-46e2-bbc9-09a000ae3f30-kube-api-access-qddd6\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:39.154252 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.154236    5704 scope.go:122] "RemoveContainer" containerID="09c3cbfdcec3d7a9720b58d695b1da75bf8dd9c5dcd034a3636390478203a680"
Oct 30 19:20:39.154497 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:20:39.154478    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09c3cbfdcec3d7a9720b58d695b1da75bf8dd9c5dcd034a3636390478203a680\": container with ID starting with 09c3cbfdcec3d7a9720b58d695b1da75bf8dd9c5dcd034a3636390478203a680 not found: ID does not exist" containerID="09c3cbfdcec3d7a9720b58d695b1da75bf8dd9c5dcd034a3636390478203a680"
Oct 30 19:20:39.154536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.154503    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c3cbfdcec3d7a9720b58d695b1da75bf8dd9c5dcd034a3636390478203a680"} err="failed to get container status \"09c3cbfdcec3d7a9720b58d695b1da75bf8dd9c5dcd034a3636390478203a680\": rpc error: code = NotFound desc = could not find container \"09c3cbfdcec3d7a9720b58d695b1da75bf8dd9c5dcd034a3636390478203a680\": container with ID starting with 09c3cbfdcec3d7a9720b58d695b1da75bf8dd9c5dcd034a3636390478203a680 not found: ID does not exist"
Oct 30 19:20:39.181955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.181923    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="930f1ac9-f21e-4f2a-ae60-43b41309469b" path="/var/lib/kubelet/pods/930f1ac9-f21e-4f2a-ae60-43b41309469b/volumes"
Oct 30 19:20:39.182364 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.182346    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e29dbede-923d-430c-895a-c28b4c89d0d7" path="/var/lib/kubelet/pods/e29dbede-923d-430c-895a-c28b4c89d0d7/volumes"
Oct 30 19:20:39.263846 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.263819    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"]
Oct 30 19:20:39.276907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.276887    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-795c67df96-mjmrf"]
Oct 30 19:20:39.381857 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.381834    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"]
Oct 30 19:20:39.384605 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.384556    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-fb4a1b-66545769f-5zp9r"]
Oct 30 19:20:39.840325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:39.840279    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:40.058663 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:40.058631    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:40.843982 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:40.843943    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:41.083218 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.083175    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:41.189440 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.189358    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="298df6c6-b7d1-46e2-bbc9-09a000ae3f30" path="/var/lib/kubelet/pods/298df6c6-b7d1-46e2-bbc9-09a000ae3f30/volumes"
Oct 30 19:20:41.189914 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.189893    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a933f15e-0fa7-4c6d-99a2-b3c4ba0250da" path="/var/lib/kubelet/pods/a933f15e-0fa7-4c6d-99a2-b3c4ba0250da/volumes"
Oct 30 19:20:41.394448 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.394419    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"]
Oct 30 19:20:41.395327 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.395306    5704 cpu_manager.go:414] "containerMap: removing container" podUID="8db7e37a-9924-4ce9-bac0-67edd0374bf0" containerName="payload-processing"
Oct 30 19:20:41.395486 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.395471    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8db7e37a-9924-4ce9-bac0-67edd0374bf0" containerName="payload-processing"
Oct 30 19:20:41.395649 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.395637    5704 cpu_manager.go:414] "containerMap: removing container" podUID="e29dbede-923d-430c-895a-c28b4c89d0d7" containerName="payload-pre-processing"
Oct 30 19:20:41.395753 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.395742    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e29dbede-923d-430c-895a-c28b4c89d0d7" containerName="payload-pre-processing"
Oct 30 19:20:41.399172 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.399155    5704 cpu_manager.go:414] "containerMap: removing container" podUID="a933f15e-0fa7-4c6d-99a2-b3c4ba0250da" containerName="payload-processing"
Oct 30 19:20:41.399331 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.399320    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a933f15e-0fa7-4c6d-99a2-b3c4ba0250da" containerName="payload-processing"
Oct 30 19:20:41.399428 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.399420    5704 cpu_manager.go:414] "containerMap: removing container" podUID="298df6c6-b7d1-46e2-bbc9-09a000ae3f30" containerName="payload-pre-processing"
Oct 30 19:20:41.399493 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.399484    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="298df6c6-b7d1-46e2-bbc9-09a000ae3f30" containerName="payload-pre-processing"
Oct 30 19:20:41.399558 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.399550    5704 cpu_manager.go:414] "containerMap: removing container" podUID="eb1c92a8-55e3-46fd-ae18-410645416414" containerName="payload-pre-processing"
Oct 30 19:20:41.399617 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.399609    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb1c92a8-55e3-46fd-ae18-410645416414" containerName="payload-pre-processing"
Oct 30 19:20:41.399676 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.399669    5704 cpu_manager.go:414] "containerMap: removing container" podUID="930f1ac9-f21e-4f2a-ae60-43b41309469b" containerName="payload-processing"
Oct 30 19:20:41.399738 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.399730    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="930f1ac9-f21e-4f2a-ae60-43b41309469b" containerName="payload-processing"
Oct 30 19:20:41.399957 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.399946    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="e29dbede-923d-430c-895a-c28b4c89d0d7" containerName="payload-pre-processing"
Oct 30 19:20:41.400050 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.400023    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="a933f15e-0fa7-4c6d-99a2-b3c4ba0250da" containerName="payload-processing"
Oct 30 19:20:41.400140 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.400131    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="298df6c6-b7d1-46e2-bbc9-09a000ae3f30" containerName="payload-pre-processing"
Oct 30 19:20:41.400204 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.400196    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb1c92a8-55e3-46fd-ae18-410645416414" containerName="payload-pre-processing"
Oct 30 19:20:41.400271 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.400263    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="930f1ac9-f21e-4f2a-ae60-43b41309469b" containerName="payload-processing"
Oct 30 19:20:41.400344 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.400337    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="8db7e37a-9924-4ce9-bac0-67edd0374bf0" containerName="payload-processing"
Oct 30 19:20:41.404855 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.404830    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"
Oct 30 19:20:41.427847 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.427822    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90a10f6f-4267-4abe-8ad2-6d4b31456575-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk\" (UID: \"90a10f6f-4267-4abe-8ad2-6d4b31456575\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"
Oct 30 19:20:41.428103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.428083    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlmkx\" (UniqueName: \"kubernetes.io/projected/90a10f6f-4267-4abe-8ad2-6d4b31456575-kube-api-access-nlmkx\") pod \"payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk\" (UID: \"90a10f6f-4267-4abe-8ad2-6d4b31456575\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"
Oct 30 19:20:41.500613 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.500588    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-65d741\""
Oct 30 19:20:41.500777 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.500622    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-65d741-dockercfg-jz2m8\""
Oct 30 19:20:41.503147 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.503124    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"]
Oct 30 19:20:41.529560 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.529533    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlmkx\" (UniqueName: \"kubernetes.io/projected/90a10f6f-4267-4abe-8ad2-6d4b31456575-kube-api-access-nlmkx\") pod \"payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk\" (UID: \"90a10f6f-4267-4abe-8ad2-6d4b31456575\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"
Oct 30 19:20:41.529687 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.529580    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90a10f6f-4267-4abe-8ad2-6d4b31456575-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk\" (UID: \"90a10f6f-4267-4abe-8ad2-6d4b31456575\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"
Oct 30 19:20:41.530303 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.530285    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90a10f6f-4267-4abe-8ad2-6d4b31456575-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk\" (UID: \"90a10f6f-4267-4abe-8ad2-6d4b31456575\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"
Oct 30 19:20:41.559555 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.559534    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"]
Oct 30 19:20:41.562875 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.562858    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"
Oct 30 19:20:41.586795 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.586776    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlmkx\" (UniqueName: \"kubernetes.io/projected/90a10f6f-4267-4abe-8ad2-6d4b31456575-kube-api-access-nlmkx\") pod \"payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk\" (UID: \"90a10f6f-4267-4abe-8ad2-6d4b31456575\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"
Oct 30 19:20:41.621426 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.621402    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"]
Oct 30 19:20:41.630770 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.630745    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsz7r\" (UniqueName: \"kubernetes.io/projected/ce342a00-bd7c-42ca-a1ae-a686c367493c-kube-api-access-qsz7r\") pod \"payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr\" (UID: \"ce342a00-bd7c-42ca-a1ae-a686c367493c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"
Oct 30 19:20:41.630918 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.630898    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce342a00-bd7c-42ca-a1ae-a686c367493c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr\" (UID: \"ce342a00-bd7c-42ca-a1ae-a686c367493c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"
Oct 30 19:20:41.725837 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.725813    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"
Oct 30 19:20:41.731622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.731597    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce342a00-bd7c-42ca-a1ae-a686c367493c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr\" (UID: \"ce342a00-bd7c-42ca-a1ae-a686c367493c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"
Oct 30 19:20:41.731777 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.731758    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsz7r\" (UniqueName: \"kubernetes.io/projected/ce342a00-bd7c-42ca-a1ae-a686c367493c-kube-api-access-qsz7r\") pod \"payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr\" (UID: \"ce342a00-bd7c-42ca-a1ae-a686c367493c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"
Oct 30 19:20:41.732302 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.732283    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce342a00-bd7c-42ca-a1ae-a686c367493c-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr\" (UID: \"ce342a00-bd7c-42ca-a1ae-a686c367493c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"
Oct 30 19:20:41.798325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.798280    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsz7r\" (UniqueName: \"kubernetes.io/projected/ce342a00-bd7c-42ca-a1ae-a686c367493c-kube-api-access-qsz7r\") pod \"payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr\" (UID: \"ce342a00-bd7c-42ca-a1ae-a686c367493c\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"
Oct 30 19:20:41.875166 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:41.875141    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"
Oct 30 19:20:42.097277 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:42.097250    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"]
Oct 30 19:20:42.099335 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:20:42.099310    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90a10f6f_4267_4abe_8ad2_6d4b31456575.slice/crio-f647d37cb2147ab6c3505d34c98ed1948c0c2cb3dca51878324be4493a67da70 WatchSource:0}: Error finding container f647d37cb2147ab6c3505d34c98ed1948c0c2cb3dca51878324be4493a67da70: Status 404 returned error can't find the container with id f647d37cb2147ab6c3505d34c98ed1948c0c2cb3dca51878324be4493a67da70
Oct 30 19:20:42.170897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:42.170869    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk" event={"ID":"90a10f6f-4267-4abe-8ad2-6d4b31456575","Type":"ContainerStarted","Data":"92cc1abf0de3ac816862142f3159d877f57e2964b752be9192ae5f6060bbd624"}
Oct 30 19:20:42.171006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:42.170916    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk" event={"ID":"90a10f6f-4267-4abe-8ad2-6d4b31456575","Type":"ContainerStarted","Data":"f647d37cb2147ab6c3505d34c98ed1948c0c2cb3dca51878324be4493a67da70"}
Oct 30 19:20:42.171121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:42.171018    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"
Oct 30 19:20:42.250579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:42.250559    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"]
Oct 30 19:20:42.252585 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:20:42.252559    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce342a00_bd7c_42ca_a1ae_a686c367493c.slice/crio-17f206ac9067bab4ec40d194696e4597243027bcb177f20e722eeab2b155f698 WatchSource:0}: Error finding container 17f206ac9067bab4ec40d194696e4597243027bcb177f20e722eeab2b155f698: Status 404 returned error can't find the container with id 17f206ac9067bab4ec40d194696e4597243027bcb177f20e722eeab2b155f698
Oct 30 19:20:42.286589 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:42.286545    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk" podStartSLOduration=1.286531549 podStartE2EDuration="1.286531549s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:42.283677544 +0000 UTC m=+2015.229845447" watchObservedRunningTime="2026-10-30 19:20:42.286531549 +0000 UTC m=+2015.232699451"
Oct 30 19:20:42.524866 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:42.524831    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:42.743253 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:42.743225    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:43.180514 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:43.180482    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"
Oct 30 19:20:43.180867 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:43.180524    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr" event={"ID":"ce342a00-bd7c-42ca-a1ae-a686c367493c","Type":"ContainerStarted","Data":"5773ca92716882bba7b52d703d9726ce42f1b5a6043a0623c657fecb8f58317e"}
Oct 30 19:20:43.180867 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:43.180544    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr" event={"ID":"ce342a00-bd7c-42ca-a1ae-a686c367493c","Type":"ContainerStarted","Data":"17f206ac9067bab4ec40d194696e4597243027bcb177f20e722eeab2b155f698"}
Oct 30 19:20:43.290076 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:43.290017    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr" podStartSLOduration=2.290003682 podStartE2EDuration="2.290003682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:43.288368094 +0000 UTC m=+2016.234535996" watchObservedRunningTime="2026-10-30 19:20:43.290003682 +0000 UTC m=+2016.236171584"
Oct 30 19:20:44.141686 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.141655    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"]
Oct 30 19:20:44.141881 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.141842    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c" podUID="01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe" containerName="maas-api" containerID="cri-o://d6c8f0b413a487fd5d031e59d93ce1479285197c887a929abc0926529f79e3d3" gracePeriod=30
Oct 30 19:20:44.311107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.311060    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:44.408634 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.408578    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:20:44.479551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.467887    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-maas-api-tls\") pod \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") "
Oct 30 19:20:44.479551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.467954    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-openshift-service-ca\") pod \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") "
Oct 30 19:20:44.479551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.468003    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-metrics-certs\" (UniqueName: \"kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-metrics-certs\") pod \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") "
Oct 30 19:20:44.479551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.468241    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-kube-api-access-mzvqd\" (UniqueName: \"kubernetes.io/projected/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-kube-api-access-mzvqd\") pod \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\" (UID: \"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe\") "
Oct 30 19:20:44.479551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.479410    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-openshift-service-ca" pod "01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe" (UID: "01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:20:44.481249 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.481195    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-kube-api-access-mzvqd" pod "01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe" (UID: "01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe"). InnerVolumeSpecName "kube-api-access-mzvqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:20:44.492450 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.491625    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-maas-api-tls" pod "01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe" (UID: "01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:20:44.492450 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.492261    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-metrics-certs" pod "01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe" (UID: "01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:20:44.570471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.570436    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:44.570471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.570464    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:44.570471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.570478    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:44.570718 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:44.570490    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzvqd\" (UniqueName: \"kubernetes.io/projected/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe-kube-api-access-mzvqd\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:20:45.268056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:45.265268    5704 generic.go:356] "Generic (PLEG): container finished" podID="01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe" containerID="d6c8f0b413a487fd5d031e59d93ce1479285197c887a929abc0926529f79e3d3" exitCode=0
Oct 30 19:20:45.268056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:45.265425    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c" event={"ID":"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe","Type":"ContainerDied","Data":"d6c8f0b413a487fd5d031e59d93ce1479285197c887a929abc0926529f79e3d3"}
Oct 30 19:20:45.268056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:45.265471    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c" event={"ID":"01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe","Type":"ContainerDied","Data":"fd565ce2f25345802dfe538a7a8eeabf23b3ff89adc59616b2d068b7bf105dcf"}
Oct 30 19:20:45.268056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:45.265503    5704 scope.go:122] "RemoveContainer" containerID="d6c8f0b413a487fd5d031e59d93ce1479285197c887a929abc0926529f79e3d3"
Oct 30 19:20:45.268056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:45.265689    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"
Oct 30 19:20:45.286301 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:45.286278    5704 scope.go:122] "RemoveContainer" containerID="d6c8f0b413a487fd5d031e59d93ce1479285197c887a929abc0926529f79e3d3"
Oct 30 19:20:45.286614 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:20:45.286594    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6c8f0b413a487fd5d031e59d93ce1479285197c887a929abc0926529f79e3d3\": container with ID starting with d6c8f0b413a487fd5d031e59d93ce1479285197c887a929abc0926529f79e3d3 not found: ID does not exist" containerID="d6c8f0b413a487fd5d031e59d93ce1479285197c887a929abc0926529f79e3d3"
Oct 30 19:20:45.286684 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:45.286623    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6c8f0b413a487fd5d031e59d93ce1479285197c887a929abc0926529f79e3d3"} err="failed to get container status \"d6c8f0b413a487fd5d031e59d93ce1479285197c887a929abc0926529f79e3d3\": rpc error: code = NotFound desc = could not find container \"d6c8f0b413a487fd5d031e59d93ce1479285197c887a929abc0926529f79e3d3\": container with ID starting with d6c8f0b413a487fd5d031e59d93ce1479285197c887a929abc0926529f79e3d3 not found: ID does not exist"
Oct 30 19:20:45.423631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:45.423594    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"]
Oct 30 19:20:45.469801 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:45.469769    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-2ec7ea-5569697464-blz7c"]
Oct 30 19:20:45.850420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:45.850382    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:46.068940 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:46.068905    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:46.270256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:46.270226    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:46.776876 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:46.776846    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:47.179642 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:47.179570    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe" path="/var/lib/kubelet/pods/01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe/volumes"
Oct 30 19:20:50.698422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.698379    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"]
Oct 30 19:20:50.699008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.698992    5704 cpu_manager.go:414] "containerMap: removing container" podUID="01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe" containerName="maas-api"
Oct 30 19:20:50.699008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.699007    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe" containerName="maas-api"
Oct 30 19:20:50.699191 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.699175    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="01f17cad-ebff-4abf-b7a6-a7dcf2baa2fe" containerName="maas-api"
Oct 30 19:20:50.702800 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.702780    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:50.740847 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.740821    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-metrics-certs\") pod \"maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:50.741009 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.740859    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cae896c-360e-492f-b987-dec361cf72e4-openshift-service-ca\") pod \"maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:50.741009 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.740894    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4pb4\" (UniqueName: \"kubernetes.io/projected/7cae896c-360e-492f-b987-dec361cf72e4-kube-api-access-c4pb4\") pod \"maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:50.741127 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.741105    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-maas-api-tls\") pod \"maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:50.751875 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.751851    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-324ebe-serving-cert\""
Oct 30 19:20:50.800043 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.800014    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"]
Oct 30 19:20:50.842376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.842346    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-maas-api-tls\") pod \"maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:50.842524 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.842450    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-metrics-certs\") pod \"maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:50.842524 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:20:50.842488    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-serving-cert: secret "maas-api-e2e-api-a-324ebe-serving-cert" not found
Oct 30 19:20:50.842874 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.842493    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cae896c-360e-492f-b987-dec361cf72e4-openshift-service-ca\") pod \"maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:50.842874 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:20:50.842597    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-maas-api-tls podName:7cae896c-360e-492f-b987-dec361cf72e4 nodeName:}" failed. No retries permitted until 2026-10-30 19:20:51.342573896 +0000 UTC m=+2024.288741799 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-maas-api-tls") pod "maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh" (UID: "7cae896c-360e-492f-b987-dec361cf72e4") : secret "maas-api-e2e-api-a-324ebe-serving-cert" not found
Oct 30 19:20:50.842874 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.842618    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4pb4\" (UniqueName: \"kubernetes.io/projected/7cae896c-360e-492f-b987-dec361cf72e4-kube-api-access-c4pb4\") pod \"maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:50.843309 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.843290    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cae896c-360e-492f-b987-dec361cf72e4-openshift-service-ca\") pod \"maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:50.845630 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.845605    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-metrics-certs\") pod \"maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:50.900753 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.900723    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4pb4\" (UniqueName: \"kubernetes.io/projected/7cae896c-360e-492f-b987-dec361cf72e4-kube-api-access-c4pb4\") pod \"maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:50.926311 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:50.926282    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:51.162479 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:51.162436    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:51.372992 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:51.372960    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-maas-api-tls\") pod \"maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:51.375330 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:51.375312    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-maas-api-tls\") pod \"maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:51.384954 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:51.384916    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:51.615958 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:51.615930    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:51.986475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:51.986450    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"]
Oct 30 19:20:51.988751 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:20:51.988725    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cae896c_360e_492f_b987_dec361cf72e4.slice/crio-e0b914fe90fe24872174077bb822061f9a5c0f2c80a043f11f99861cc1fd6e27 WatchSource:0}: Error finding container e0b914fe90fe24872174077bb822061f9a5c0f2c80a043f11f99861cc1fd6e27: Status 404 returned error can't find the container with id e0b914fe90fe24872174077bb822061f9a5c0f2c80a043f11f99861cc1fd6e27
Oct 30 19:20:52.312352 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:52.312314    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh" event={"ID":"7cae896c-360e-492f-b987-dec361cf72e4","Type":"ContainerStarted","Data":"e0b914fe90fe24872174077bb822061f9a5c0f2c80a043f11f99861cc1fd6e27"}
Oct 30 19:20:53.183001 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:53.182925    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"
Oct 30 19:20:53.320376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:53.320345    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh" event={"ID":"7cae896c-360e-492f-b987-dec361cf72e4","Type":"ContainerStarted","Data":"2bd5c17246c59f16d9135de9e71196240dd7d5383bc40ff204bc6dbd52780d71"}
Oct 30 19:20:53.320564 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:53.320423    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:53.405727 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:53.405693    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:53.599286 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:53.599252    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:53.788503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:53.788470    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:54.188848 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:54.188778    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"
Oct 30 19:20:54.300187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:54.300139    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh" podStartSLOduration=3.529879174 podStartE2EDuration="4.300126391s" totalImagesPullingTime="770.247217ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:20:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:20:53.432334751 +0000 UTC m=+2026.378502657" watchObservedRunningTime="2026-10-30 19:20:54.300126391 +0000 UTC m=+2027.246294294"
Oct 30 19:20:54.817013 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:54.816975    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:55.030659 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:55.030626    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:20:55.476992 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:55.476964    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:55.829892 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:55.829815    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:56.023564 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:56.023525    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:59.331509 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:59.331484    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:20:59.513636 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:59.513603    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:20:59.967296 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:20:59.967260    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:00.206606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:00.206574    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:01.128587 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:01.128559    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"]
Oct 30 19:21:01.132214 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:01.132191    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"
Oct 30 19:21:01.182233 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:01.182206    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-905566\""
Oct 30 19:21:01.222738 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:01.222703    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5c5d61a-a137-469c-8da5-3ceb9596ece2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq\" (UID: \"c5c5d61a-a137-469c-8da5-3ceb9596ece2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"
Oct 30 19:21:01.222909 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:01.222891    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7gkh\" (UniqueName: \"kubernetes.io/projected/c5c5d61a-a137-469c-8da5-3ceb9596ece2-kube-api-access-m7gkh\") pod \"payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq\" (UID: \"c5c5d61a-a137-469c-8da5-3ceb9596ece2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"
Oct 30 19:21:01.227238 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:01.227216    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-905566-dockercfg-rvjsn\""
Oct 30 19:21:01.229750 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:01.229726    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"]
Oct 30 19:21:01.323921 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:01.323898    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7gkh\" (UniqueName: \"kubernetes.io/projected/c5c5d61a-a137-469c-8da5-3ceb9596ece2-kube-api-access-m7gkh\") pod \"payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq\" (UID: \"c5c5d61a-a137-469c-8da5-3ceb9596ece2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"
Oct 30 19:21:01.324080 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:01.323964    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5c5d61a-a137-469c-8da5-3ceb9596ece2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq\" (UID: \"c5c5d61a-a137-469c-8da5-3ceb9596ece2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"
Oct 30 19:21:01.324642 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:01.324623    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5c5d61a-a137-469c-8da5-3ceb9596ece2-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq\" (UID: \"c5c5d61a-a137-469c-8da5-3ceb9596ece2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"
Oct 30 19:21:01.381605 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:01.381549    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7gkh\" (UniqueName: \"kubernetes.io/projected/c5c5d61a-a137-469c-8da5-3ceb9596ece2-kube-api-access-m7gkh\") pod \"payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq\" (UID: \"c5c5d61a-a137-469c-8da5-3ceb9596ece2\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"
Oct 30 19:21:01.445807 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:01.445785    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"
Oct 30 19:21:01.819188 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:01.819165    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"]
Oct 30 19:21:01.821571 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:21:01.821542    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5c5d61a_a137_469c_8da5_3ceb9596ece2.slice/crio-5588fc5d59a5676ef3eb8a7f3fcc50c20a8208c6e05631c6602ac6196251eb7b WatchSource:0}: Error finding container 5588fc5d59a5676ef3eb8a7f3fcc50c20a8208c6e05631c6602ac6196251eb7b: Status 404 returned error can't find the container with id 5588fc5d59a5676ef3eb8a7f3fcc50c20a8208c6e05631c6602ac6196251eb7b
Oct 30 19:21:02.391961 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:02.391924    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq" event={"ID":"c5c5d61a-a137-469c-8da5-3ceb9596ece2","Type":"ContainerStarted","Data":"5588fc5d59a5676ef3eb8a7f3fcc50c20a8208c6e05631c6602ac6196251eb7b"}
Oct 30 19:21:02.802078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:02.802016    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:03.115883 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.115797    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:03.287560 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.287294    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:03.458396 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:21:03.458265    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:21:03.635315 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.635279    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"]
Oct 30 19:21:03.650063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.646874    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"
Oct 30 19:21:03.690057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.684892    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx46t\" (UniqueName: \"kubernetes.io/projected/af419107-fbad-4021-88a0-aef9c988537e-kube-api-access-jx46t\") pod \"payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx\" (UID: \"af419107-fbad-4021-88a0-aef9c988537e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"
Oct 30 19:21:03.690057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.685137    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af419107-fbad-4021-88a0-aef9c988537e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx\" (UID: \"af419107-fbad-4021-88a0-aef9c988537e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"
Oct 30 19:21:03.750635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.750103    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-7a790b-dockercfg-2zml9\""
Oct 30 19:21:03.763225 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.763195    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-7a790b\""
Oct 30 19:21:03.764874 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.764837    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"]
Oct 30 19:21:03.786247 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.786217    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jx46t\" (UniqueName: \"kubernetes.io/projected/af419107-fbad-4021-88a0-aef9c988537e-kube-api-access-jx46t\") pod \"payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx\" (UID: \"af419107-fbad-4021-88a0-aef9c988537e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"
Oct 30 19:21:03.786411 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.786399    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af419107-fbad-4021-88a0-aef9c988537e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx\" (UID: \"af419107-fbad-4021-88a0-aef9c988537e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"
Oct 30 19:21:03.787177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.787152    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af419107-fbad-4021-88a0-aef9c988537e-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx\" (UID: \"af419107-fbad-4021-88a0-aef9c988537e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"
Oct 30 19:21:03.808716 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.808681    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"]
Oct 30 19:21:03.814107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.813155    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"
Oct 30 19:21:03.844011 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.843992    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx46t\" (UniqueName: \"kubernetes.io/projected/af419107-fbad-4021-88a0-aef9c988537e-kube-api-access-jx46t\") pod \"payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx\" (UID: \"af419107-fbad-4021-88a0-aef9c988537e\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"
Oct 30 19:21:03.871972 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.871952    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"]
Oct 30 19:21:03.887638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.887615    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d681bc0-72b9-4624-8978-48afb725d311-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d\" (UID: \"6d681bc0-72b9-4624-8978-48afb725d311\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"
Oct 30 19:21:03.887759 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.887738    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbljp\" (UniqueName: \"kubernetes.io/projected/6d681bc0-72b9-4624-8978-48afb725d311-kube-api-access-tbljp\") pod \"payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d\" (UID: \"6d681bc0-72b9-4624-8978-48afb725d311\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"
Oct 30 19:21:03.989489 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.989460    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d681bc0-72b9-4624-8978-48afb725d311-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d\" (UID: \"6d681bc0-72b9-4624-8978-48afb725d311\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"
Oct 30 19:21:03.989630 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.989546    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbljp\" (UniqueName: \"kubernetes.io/projected/6d681bc0-72b9-4624-8978-48afb725d311-kube-api-access-tbljp\") pod \"payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d\" (UID: \"6d681bc0-72b9-4624-8978-48afb725d311\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"
Oct 30 19:21:03.990251 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:03.990233    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d681bc0-72b9-4624-8978-48afb725d311-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d\" (UID: \"6d681bc0-72b9-4624-8978-48afb725d311\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"
Oct 30 19:21:04.009908 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:04.009852    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"
Oct 30 19:21:04.047747 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:04.047724    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbljp\" (UniqueName: \"kubernetes.io/projected/6d681bc0-72b9-4624-8978-48afb725d311-kube-api-access-tbljp\") pod \"payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d\" (UID: \"6d681bc0-72b9-4624-8978-48afb725d311\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"
Oct 30 19:21:04.127782 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:04.127741    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"
Oct 30 19:21:04.381577 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:04.381552    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"]
Oct 30 19:21:04.382988 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:21:04.382962    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf419107_fbad_4021_88a0_aef9c988537e.slice/crio-096ae654dc53c4a552e501e5b2a5edff7496c0db1cfe8f25d3fb0f1ccd87736c WatchSource:0}: Error finding container 096ae654dc53c4a552e501e5b2a5edff7496c0db1cfe8f25d3fb0f1ccd87736c: Status 404 returned error can't find the container with id 096ae654dc53c4a552e501e5b2a5edff7496c0db1cfe8f25d3fb0f1ccd87736c
Oct 30 19:21:04.425985 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:04.425960    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx" event={"ID":"af419107-fbad-4021-88a0-aef9c988537e","Type":"ContainerStarted","Data":"096ae654dc53c4a552e501e5b2a5edff7496c0db1cfe8f25d3fb0f1ccd87736c"}
Oct 30 19:21:04.502778 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:04.502755    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"]
Oct 30 19:21:04.505082 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:21:04.505057    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d681bc0_72b9_4624_8978_48afb725d311.slice/crio-7803c625db7a5ed155dc51fcfe3847703c4ee0c05900501642bc6c42ff823d79 WatchSource:0}: Error finding container 7803c625db7a5ed155dc51fcfe3847703c4ee0c05900501642bc6c42ff823d79: Status 404 returned error can't find the container with id 7803c625db7a5ed155dc51fcfe3847703c4ee0c05900501642bc6c42ff823d79
Oct 30 19:21:05.433986 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:05.433949    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d" event={"ID":"6d681bc0-72b9-4624-8978-48afb725d311","Type":"ContainerStarted","Data":"b7311c6314393d7d385bc51f4026f5106cb9c6d13f1da0860da269e26ca5448a"}
Oct 30 19:21:05.434158 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:05.434011    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d" event={"ID":"6d681bc0-72b9-4624-8978-48afb725d311","Type":"ContainerStarted","Data":"7803c625db7a5ed155dc51fcfe3847703c4ee0c05900501642bc6c42ff823d79"}
Oct 30 19:21:05.434158 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:05.434058    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"
Oct 30 19:21:05.435619 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:05.435594    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx" event={"ID":"af419107-fbad-4021-88a0-aef9c988537e","Type":"ContainerStarted","Data":"0c2a5c7cafec01804dacd58e21fcff9d20b3fced08e2769aa195b3e24562cf3c"}
Oct 30 19:21:05.435743 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:05.435673    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"
Oct 30 19:21:05.548387 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:05.548335    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d" podStartSLOduration=2.548317805 podStartE2EDuration="2.548317805s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:05.546628888 +0000 UTC m=+2038.492796791" watchObservedRunningTime="2026-10-30 19:21:05.548317805 +0000 UTC m=+2038.494485710"
Oct 30 19:21:05.687598 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:05.684366    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx" podStartSLOduration=2.684343816 podStartE2EDuration="2.684343816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:05.681766976 +0000 UTC m=+2038.627934881" watchObservedRunningTime="2026-10-30 19:21:05.684343816 +0000 UTC m=+2038.630511723"
Oct 30 19:21:05.845060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:05.844130    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:06.452564 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:06.452471    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq" event={"ID":"c5c5d61a-a137-469c-8da5-3ceb9596ece2","Type":"ContainerStarted","Data":"5e8c35c540976263ee3b0648ff8c39760222fc3c33118475894a4e7fb29a8b94"}
Oct 30 19:21:06.452837 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:06.452816    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"
Oct 30 19:21:06.563538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:06.563489    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq" podStartSLOduration=1.18776056 podStartE2EDuration="5.563474506s" totalImagesPullingTime="4.375713946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:06.561534667 +0000 UTC m=+2039.507702572" watchObservedRunningTime="2026-10-30 19:21:06.563474506 +0000 UTC m=+2039.509642408"
Oct 30 19:21:07.728971 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:07.728943    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:12.735065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:12.728748    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:16.198059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:16.196331    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:16.456422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:16.456341    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"
Oct 30 19:21:16.456422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:16.456424    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"
Oct 30 19:21:16.606312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:16.606268    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:17.247353 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.247317    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"]
Oct 30 19:21:17.252293 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.252269    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"
Oct 30 19:21:17.301024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.301001    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-4ec347\""
Oct 30 19:21:17.347230 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.347200    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-4ec347-dockercfg-9r48c\""
Oct 30 19:21:17.349814 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.349792    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"]
Oct 30 19:21:17.358983 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.358958    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-545pp\" (UniqueName: \"kubernetes.io/projected/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-kube-api-access-545pp\") pod \"payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn\" (UID: \"d0289572-5d8d-4e2f-a14d-4bd58d050bd7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"
Oct 30 19:21:17.359152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.359132    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn\" (UID: \"d0289572-5d8d-4e2f-a14d-4bd58d050bd7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"
Oct 30 19:21:17.460120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.460086    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-545pp\" (UniqueName: \"kubernetes.io/projected/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-kube-api-access-545pp\") pod \"payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn\" (UID: \"d0289572-5d8d-4e2f-a14d-4bd58d050bd7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"
Oct 30 19:21:17.460283 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.460210    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn\" (UID: \"d0289572-5d8d-4e2f-a14d-4bd58d050bd7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"
Oct 30 19:21:17.460900 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.460877    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn\" (UID: \"d0289572-5d8d-4e2f-a14d-4bd58d050bd7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"
Oct 30 19:21:17.461925 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.461899    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"
Oct 30 19:21:17.517693 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.517626    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-545pp\" (UniqueName: \"kubernetes.io/projected/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-kube-api-access-545pp\") pod \"payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn\" (UID: \"d0289572-5d8d-4e2f-a14d-4bd58d050bd7\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"
Oct 30 19:21:17.565815 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.565771    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"
Oct 30 19:21:17.692582 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.692550    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"]
Oct 30 19:21:17.696747 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.696726    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"
Oct 30 19:21:17.752544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.752518    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"]
Oct 30 19:21:17.764832 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.764789    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/353a50b4-be00-45b2-a6b1-af6498c0af69-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9\" (UID: \"353a50b4-be00-45b2-a6b1-af6498c0af69\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"
Oct 30 19:21:17.765001 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.764981    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnv5c\" (UniqueName: \"kubernetes.io/projected/353a50b4-be00-45b2-a6b1-af6498c0af69-kube-api-access-bnv5c\") pod \"payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9\" (UID: \"353a50b4-be00-45b2-a6b1-af6498c0af69\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"
Oct 30 19:21:17.866262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.866189    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/353a50b4-be00-45b2-a6b1-af6498c0af69-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9\" (UID: \"353a50b4-be00-45b2-a6b1-af6498c0af69\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"
Oct 30 19:21:17.866420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.866325    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnv5c\" (UniqueName: \"kubernetes.io/projected/353a50b4-be00-45b2-a6b1-af6498c0af69-kube-api-access-bnv5c\") pod \"payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9\" (UID: \"353a50b4-be00-45b2-a6b1-af6498c0af69\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"
Oct 30 19:21:17.866830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.866808    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/353a50b4-be00-45b2-a6b1-af6498c0af69-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9\" (UID: \"353a50b4-be00-45b2-a6b1-af6498c0af69\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"
Oct 30 19:21:17.923649 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.923622    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnv5c\" (UniqueName: \"kubernetes.io/projected/353a50b4-be00-45b2-a6b1-af6498c0af69-kube-api-access-bnv5c\") pod \"payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9\" (UID: \"353a50b4-be00-45b2-a6b1-af6498c0af69\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"
Oct 30 19:21:17.943412 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:17.943393    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"]
Oct 30 19:21:17.945348 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:21:17.945320    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0289572_5d8d_4e2f_a14d_4bd58d050bd7.slice/crio-02177cf2e9d39532b696e341684c86c014b2bb1dee8a4ba2fd11263f53aeff2d WatchSource:0}: Error finding container 02177cf2e9d39532b696e341684c86c014b2bb1dee8a4ba2fd11263f53aeff2d: Status 404 returned error can't find the container with id 02177cf2e9d39532b696e341684c86c014b2bb1dee8a4ba2fd11263f53aeff2d
Oct 30 19:21:18.011867 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:18.011843    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"
Oct 30 19:21:18.385829 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:18.385803    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"]
Oct 30 19:21:18.388560 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:21:18.388533    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod353a50b4_be00_45b2_a6b1_af6498c0af69.slice/crio-59c116447cc6ed0dc98afa7e9c1774e975c18c9dbc79c723b6ff6083ce0af672 WatchSource:0}: Error finding container 59c116447cc6ed0dc98afa7e9c1774e975c18c9dbc79c723b6ff6083ce0af672: Status 404 returned error can't find the container with id 59c116447cc6ed0dc98afa7e9c1774e975c18c9dbc79c723b6ff6083ce0af672
Oct 30 19:21:18.531407 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:18.531372    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9" event={"ID":"353a50b4-be00-45b2-a6b1-af6498c0af69","Type":"ContainerStarted","Data":"8d46d1e4dccf6358a17e541be48b88dc3b2418ec6ac699e4fcebebe311b25a11"}
Oct 30 19:21:18.531579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:18.531488    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"
Oct 30 19:21:18.531579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:18.531517    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9" event={"ID":"353a50b4-be00-45b2-a6b1-af6498c0af69","Type":"ContainerStarted","Data":"59c116447cc6ed0dc98afa7e9c1774e975c18c9dbc79c723b6ff6083ce0af672"}
Oct 30 19:21:18.533087 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:18.533061    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn" event={"ID":"d0289572-5d8d-4e2f-a14d-4bd58d050bd7","Type":"ContainerStarted","Data":"947c5e86d15d7b492ee32374dcf46944ead98dc7606315409f48d58f41c03051"}
Oct 30 19:21:18.533210 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:18.533115    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"
Oct 30 19:21:18.533210 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:18.533142    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn" event={"ID":"d0289572-5d8d-4e2f-a14d-4bd58d050bd7","Type":"ContainerStarted","Data":"02177cf2e9d39532b696e341684c86c014b2bb1dee8a4ba2fd11263f53aeff2d"}
Oct 30 19:21:18.644213 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:18.644123    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9" podStartSLOduration=1.6441057799999999 podStartE2EDuration="1.64410578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:18.640209839 +0000 UTC m=+2051.586377743" watchObservedRunningTime="2026-10-30 19:21:18.64410578 +0000 UTC m=+2051.590273685"
Oct 30 19:21:18.773129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:18.771554    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn" podStartSLOduration=1.771533649 podStartE2EDuration="1.771533649s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:18.751275815 +0000 UTC m=+2051.697443721" watchObservedRunningTime="2026-10-30 19:21:18.771533649 +0000 UTC m=+2051.717701555"
Oct 30 19:21:19.363087 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:19.362110    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:19.813834 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:19.813804    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:21:19.927577 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:19.927545    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"]
Oct 30 19:21:19.927791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:19.927767    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerName="istio-proxy" containerID="cri-o://222d7f606a99067fefda53971f15afd26ff33438ce59bc6594e910e4af4a60f7" gracePeriod=30
Oct 30 19:21:22.845498 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:22.845462    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:23.307309 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:23.307278    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:23.512059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:23.511185    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:23.944990 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:23.944953    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"]
Oct 30 19:21:23.949212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:23.949193    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"
Oct 30 19:21:23.991916 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:23.991882    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"]
Oct 30 19:21:24.004387 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.004367    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-fb4a1b-dockercfg-nr2qg\""
Oct 30 19:21:24.004525 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.004397    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-fb4a1b\""
Oct 30 19:21:24.013662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.013625    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxzvf\" (UniqueName: \"kubernetes.io/projected/60815cd9-d702-4e39-a59b-763096934ca6-kube-api-access-vxzvf\") pod \"payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj\" (UID: \"60815cd9-d702-4e39-a59b-763096934ca6\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"
Oct 30 19:21:24.013807 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.013786    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60815cd9-d702-4e39-a59b-763096934ca6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj\" (UID: \"60815cd9-d702-4e39-a59b-763096934ca6\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"
Oct 30 19:21:24.062862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.062824    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:24.114941 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.114915    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60815cd9-d702-4e39-a59b-763096934ca6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj\" (UID: \"60815cd9-d702-4e39-a59b-763096934ca6\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"
Oct 30 19:21:24.115143 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.115127    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxzvf\" (UniqueName: \"kubernetes.io/projected/60815cd9-d702-4e39-a59b-763096934ca6-kube-api-access-vxzvf\") pod \"payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj\" (UID: \"60815cd9-d702-4e39-a59b-763096934ca6\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"
Oct 30 19:21:24.115734 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.115714    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60815cd9-d702-4e39-a59b-763096934ca6-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj\" (UID: \"60815cd9-d702-4e39-a59b-763096934ca6\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"
Oct 30 19:21:24.194055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.189766    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxzvf\" (UniqueName: \"kubernetes.io/projected/60815cd9-d702-4e39-a59b-763096934ca6-kube-api-access-vxzvf\") pod \"payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj\" (UID: \"60815cd9-d702-4e39-a59b-763096934ca6\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"
Oct 30 19:21:24.267098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.264256    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"
Oct 30 19:21:24.340159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.338847    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:24.497100 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.497066    5704 patch_prober.go:29] interesting pod/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:21:24.497258 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.497121    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:21:24.709721 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:24.709691    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"]
Oct 30 19:21:24.712238 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:21:24.712212    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60815cd9_d702_4e39_a59b_763096934ca6.slice/crio-037b91811b5fef11d4f702fbbc4fca2dff6feb1cd65cc58d796257befbdd70c4 WatchSource:0}: Error finding container 037b91811b5fef11d4f702fbbc4fca2dff6feb1cd65cc58d796257befbdd70c4: Status 404 returned error can't find the container with id 037b91811b5fef11d4f702fbbc4fca2dff6feb1cd65cc58d796257befbdd70c4
Oct 30 19:21:25.265082 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.261930    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:21:25.365425 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.365330    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-data\") pod \"f4827e66-1986-4644-bfa8-9741b5c53cd3\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") "
Oct 30 19:21:25.365604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.365462    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-envoy\") pod \"f4827e66-1986-4644-bfa8-9741b5c53cd3\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") "
Oct 30 19:21:25.365604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.365509    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-token\" (UniqueName: \"kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-token\") pod \"f4827e66-1986-4644-bfa8-9741b5c53cd3\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") "
Oct 30 19:21:25.365604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.365554    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-certs\") pod \"f4827e66-1986-4644-bfa8-9741b5c53cd3\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") "
Oct 30 19:21:25.365764 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.365606    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f4827e66-1986-4644-bfa8-9741b5c53cd3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4827e66-1986-4644-bfa8-9741b5c53cd3-istiod-ca-cert\") pod \"f4827e66-1986-4644-bfa8-9741b5c53cd3\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") "
Oct 30 19:21:25.365764 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.365599    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-data" pod "f4827e66-1986-4644-bfa8-9741b5c53cd3" (UID: "f4827e66-1986-4644-bfa8-9741b5c53cd3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:25.365764 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.365738    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-podinfo\") pod \"f4827e66-1986-4644-bfa8-9741b5c53cd3\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") "
Oct 30 19:21:25.365917 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.365782    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-credential-socket\") pod \"f4827e66-1986-4644-bfa8-9741b5c53cd3\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") "
Oct 30 19:21:25.365917 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.365783    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-certs" pod "f4827e66-1986-4644-bfa8-9741b5c53cd3" (UID: "f4827e66-1986-4644-bfa8-9741b5c53cd3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:25.365917 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.365834    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-socket\") pod \"f4827e66-1986-4644-bfa8-9741b5c53cd3\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") "
Oct 30 19:21:25.366089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.365931    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-kube-api-access-fsjr6\" (UniqueName: \"kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-kube-api-access-fsjr6\") pod \"f4827e66-1986-4644-bfa8-9741b5c53cd3\" (UID: \"f4827e66-1986-4644-bfa8-9741b5c53cd3\") "
Oct 30 19:21:25.366089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.365947    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4827e66-1986-4644-bfa8-9741b5c53cd3-istiod-ca-cert" pod "f4827e66-1986-4644-bfa8-9741b5c53cd3" (UID: "f4827e66-1986-4644-bfa8-9741b5c53cd3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:21:25.366089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.365990    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-credential-socket" pod "f4827e66-1986-4644-bfa8-9741b5c53cd3" (UID: "f4827e66-1986-4644-bfa8-9741b5c53cd3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:25.366089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.366044    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-socket" pod "f4827e66-1986-4644-bfa8-9741b5c53cd3" (UID: "f4827e66-1986-4644-bfa8-9741b5c53cd3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:25.366478 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.366455    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:25.366607 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.366480    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:25.366607 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.366495    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:25.366607 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.366509    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:25.366607 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.366521    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f4827e66-1986-4644-bfa8-9741b5c53cd3-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:25.368226 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.368202    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-podinfo" pod "f4827e66-1986-4644-bfa8-9741b5c53cd3" (UID: "f4827e66-1986-4644-bfa8-9741b5c53cd3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:21:25.368380 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.368362    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-envoy" pod "f4827e66-1986-4644-bfa8-9741b5c53cd3" (UID: "f4827e66-1986-4644-bfa8-9741b5c53cd3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:21:25.369132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.369111    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-kube-api-access-fsjr6" pod "f4827e66-1986-4644-bfa8-9741b5c53cd3" (UID: "f4827e66-1986-4644-bfa8-9741b5c53cd3"). InnerVolumeSpecName "kube-api-access-fsjr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:25.369207 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.369132    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-token" pod "f4827e66-1986-4644-bfa8-9741b5c53cd3" (UID: "f4827e66-1986-4644-bfa8-9741b5c53cd3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:21:25.467952 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.467911    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fsjr6\" (UniqueName: \"kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-kube-api-access-fsjr6\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:25.467952 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.467949    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:25.468241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.467963    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:25.468241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.467977    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f4827e66-1986-4644-bfa8-9741b5c53cd3-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:21:25.610830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.610792    5704 generic.go:356] "Generic (PLEG): container finished" podID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerID="222d7f606a99067fefda53971f15afd26ff33438ce59bc6594e910e4af4a60f7" exitCode=0
Oct 30 19:21:25.611005 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.610840    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"
Oct 30 19:21:25.611005 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.610990    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" event={"ID":"f4827e66-1986-4644-bfa8-9741b5c53cd3","Type":"ContainerDied","Data":"222d7f606a99067fefda53971f15afd26ff33438ce59bc6594e910e4af4a60f7"}
Oct 30 19:21:25.611112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.611085    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn" event={"ID":"f4827e66-1986-4644-bfa8-9741b5c53cd3","Type":"ContainerDied","Data":"193b883e7beb3b5ce9c8d8e8ee3b1d4de987104f1804f403334a9335cd621b57"}
Oct 30 19:21:25.611155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.611123    5704 scope.go:122] "RemoveContainer" containerID="222d7f606a99067fefda53971f15afd26ff33438ce59bc6594e910e4af4a60f7"
Oct 30 19:21:25.613746 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.613712    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj" event={"ID":"60815cd9-d702-4e39-a59b-763096934ca6","Type":"ContainerStarted","Data":"144c673dd6ae48c606d5493dea96f3fe4c2d64d83e8b721e4cca50a54d6515ea"}
Oct 30 19:21:25.614117 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.613763    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj" event={"ID":"60815cd9-d702-4e39-a59b-763096934ca6","Type":"ContainerStarted","Data":"037b91811b5fef11d4f702fbbc4fca2dff6feb1cd65cc58d796257befbdd70c4"}
Oct 30 19:21:25.614117 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.613812    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"
Oct 30 19:21:25.657939 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.657915    5704 scope.go:122] "RemoveContainer" containerID="222d7f606a99067fefda53971f15afd26ff33438ce59bc6594e910e4af4a60f7"
Oct 30 19:21:25.658335 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:21:25.658310    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"222d7f606a99067fefda53971f15afd26ff33438ce59bc6594e910e4af4a60f7\": container with ID starting with 222d7f606a99067fefda53971f15afd26ff33438ce59bc6594e910e4af4a60f7 not found: ID does not exist" containerID="222d7f606a99067fefda53971f15afd26ff33438ce59bc6594e910e4af4a60f7"
Oct 30 19:21:25.658409 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.658343    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"222d7f606a99067fefda53971f15afd26ff33438ce59bc6594e910e4af4a60f7"} err="failed to get container status \"222d7f606a99067fefda53971f15afd26ff33438ce59bc6594e910e4af4a60f7\": rpc error: code = NotFound desc = could not find container \"222d7f606a99067fefda53971f15afd26ff33438ce59bc6594e910e4af4a60f7\": container with ID starting with 222d7f606a99067fefda53971f15afd26ff33438ce59bc6594e910e4af4a60f7 not found: ID does not exist"
Oct 30 19:21:25.765779 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.765745    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"]
Oct 30 19:21:25.773113 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.773074    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-7vbkn"]
Oct 30 19:21:25.878177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.878120    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj" podStartSLOduration=2.878102166 podStartE2EDuration="2.878102166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:21:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:21:25.877198022 +0000 UTC m=+2058.823365951" watchObservedRunningTime="2026-10-30 19:21:25.878102166 +0000 UTC m=+2058.824270070"
Oct 30 19:21:25.952871 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:25.952778    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:27.182296 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:27.182212    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" path="/var/lib/kubelet/pods/f4827e66-1986-4644-bfa8-9741b5c53cd3/volumes"
Oct 30 19:21:28.281336 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:28.281298    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:28.538392 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:28.538312    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:29.540524 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:29.540495    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"
Oct 30 19:21:29.540981 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:29.540565    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"
Oct 30 19:21:30.671590 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:30.671493    5704 generic.go:356] "Generic (PLEG): container finished" podID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerID="ed9bcd6f85b66884125389f3c3ba006b2ca67c240a175ef000383d338498e017" exitCode=0
Oct 30 19:21:30.672182 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:30.671580    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" event={"ID":"096b94a8-ca2d-419e-9ce4-3554b762cbc8","Type":"ContainerDied","Data":"ed9bcd6f85b66884125389f3c3ba006b2ca67c240a175ef000383d338498e017"}
Oct 30 19:21:30.672182 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:30.672086    5704 scope.go:122] "RemoveContainer" containerID="ed9bcd6f85b66884125389f3c3ba006b2ca67c240a175ef000383d338498e017"
Oct 30 19:21:30.673627 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:30.673593    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:21:30.673709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:30.673666    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:21:30.673709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:30.673702    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:21:30.766151 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:30.766117    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:31.413662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:31.413626    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:31.605560 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:31.605526    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:31.735110 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:31.734925    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" event={"ID":"096b94a8-ca2d-419e-9ce4-3554b762cbc8","Type":"ContainerStarted","Data":"6973a05e427a9516818d575f83a23efda5188efea7dd6fd703c47fe0d572e530"}
Oct 30 19:21:32.050315 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:32.050131    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:32.089016 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:32.088982    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:21:32.089907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:32.089884    5704 patch_prober.go:29] interesting pod/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 19:21:32.090060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:32.089938    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 19:21:32.697988 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:32.697946    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:32.806059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:32.805694    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:21:33.074058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:33.074002    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:33.090108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:33.090072    5704 patch_prober.go:29] interesting pod/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 19:21:33.090274 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:33.090144    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 19:21:33.440122 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:21:33.438474    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:21:34.090095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:34.090026    5704 patch_prober.go:29] interesting pod/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 19:21:34.090516 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:34.090114    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 19:21:35.089857 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:35.089821    5704 patch_prober.go:29] interesting pod/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 19:21:35.090399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:35.089888    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 19:21:35.292232 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:35.292201    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:35.532471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:35.530455    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:35.716921 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:35.716882    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:36.089879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:36.089842    5704 patch_prober.go:29] interesting pod/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused" start-of-body=
Oct 30 19:21:36.090098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:36.089897    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.107:15021/healthz/ready\": dial tcp 10.134.0.107:15021: connect: connection refused"
Oct 30 19:21:36.647220 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:36.647188    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"
Oct 30 19:21:37.149146 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:37.149101    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:37.224650 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:37.224620    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:21:37.229011 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:37.228983    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:21:37.334850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:37.334815    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:37.517924 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:37.517892    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:40.288575 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:40.288532    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:40.493948 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:40.493893    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:40.682546 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:40.682459    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:45.282398 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:45.282364    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:49.779337 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:49.779249    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:21:49.992663 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:49.992628    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:54.923317 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:54.923280    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:56.468333 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:56.468284    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:57.338764 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:57.338724    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:57.963581 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:57.963533    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:21:58.208593 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:21:58.208556    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:01.148107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.148076    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"]
Oct 30 19:22:01.148923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.148902    5704 cpu_manager.go:414] "containerMap: removing container" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerName="istio-proxy"
Oct 30 19:22:01.149059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.148924    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerName="istio-proxy"
Oct 30 19:22:01.149137 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.149127    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="f4827e66-1986-4644-bfa8-9741b5c53cd3" containerName="istio-proxy"
Oct 30 19:22:01.151751 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.151729    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:01.184127 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.184097    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:01.184228 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.184135    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5357c9cd-0b77-44e6-930a-2b1f26884f9a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:01.184228 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.184215    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9425\" (UniqueName: \"kubernetes.io/projected/5357c9cd-0b77-44e6-930a-2b1f26884f9a-kube-api-access-d9425\") pod \"maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:01.184412 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.184399    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-metrics-certs\") pod \"maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:01.200544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.200526    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-7a790b-serving-cert\""
Oct 30 19:22:01.249527 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.249499    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"]
Oct 30 19:22:01.285764 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.285741    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-metrics-certs\") pod \"maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:01.285909 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.285815    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:01.285909 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.285839    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5357c9cd-0b77-44e6-930a-2b1f26884f9a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:01.285909 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.285864    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9425\" (UniqueName: \"kubernetes.io/projected/5357c9cd-0b77-44e6-930a-2b1f26884f9a-kube-api-access-d9425\") pod \"maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:01.286104 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:22:01.285948    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-serving-cert: secret "maas-api-e2e-ipp-a-7a790b-serving-cert" not found
Oct 30 19:22:01.286104 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:22:01.286009    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-maas-api-tls podName:5357c9cd-0b77-44e6-930a-2b1f26884f9a nodeName:}" failed. No retries permitted until 2026-10-30 19:22:01.785994781 +0000 UTC m=+2094.732162666 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-maas-api-tls") pod "maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74" (UID: "5357c9cd-0b77-44e6-930a-2b1f26884f9a") : secret "maas-api-e2e-ipp-a-7a790b-serving-cert" not found
Oct 30 19:22:01.286603 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.286583    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5357c9cd-0b77-44e6-930a-2b1f26884f9a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:01.288196 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.288179    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-metrics-certs\") pod \"maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:01.350657 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.350626    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9425\" (UniqueName: \"kubernetes.io/projected/5357c9cd-0b77-44e6-930a-2b1f26884f9a-kube-api-access-d9425\") pod \"maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:01.428778 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.428706    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:01.621062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.617526    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:01.793742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.793708    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:01.796165 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:01.796144    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:02.064227 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:02.064150    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:02.439081 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:02.439027    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"]
Oct 30 19:22:02.440984 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:22:02.440955    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5357c9cd_0b77_44e6_930a_2b1f26884f9a.slice/crio-3a93bd0f70e64f66c4f04b04ebd42fd53b8b8b75c7575b23e9a8292c2d822b85 WatchSource:0}: Error finding container 3a93bd0f70e64f66c4f04b04ebd42fd53b8b8b75c7575b23e9a8292c2d822b85: Status 404 returned error can't find the container with id 3a93bd0f70e64f66c4f04b04ebd42fd53b8b8b75c7575b23e9a8292c2d822b85
Oct 30 19:22:03.045665 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:03.045634    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:03.148633 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:03.148601    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74" event={"ID":"5357c9cd-0b77-44e6-930a-2b1f26884f9a","Type":"ContainerStarted","Data":"3a93bd0f70e64f66c4f04b04ebd42fd53b8b8b75c7575b23e9a8292c2d822b85"}
Oct 30 19:22:03.251536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:03.251503    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:03.486068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:03.485058    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:03.499421 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:22:03.499381    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:22:04.156544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:04.156506    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74" event={"ID":"5357c9cd-0b77-44e6-930a-2b1f26884f9a","Type":"ContainerStarted","Data":"d8c9b10539368e0dada0e94aa76c573f2e785a6e658d05dc596936cbc79dc0f7"}
Oct 30 19:22:04.156705 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:04.156592    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:04.276759 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:04.276712    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74" podStartSLOduration=2.404732587 podStartE2EDuration="3.276697451s" totalImagesPullingTime="871.964864ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:04.276150103 +0000 UTC m=+2097.222318009" watchObservedRunningTime="2026-10-30 19:22:04.276697451 +0000 UTC m=+2097.222865353"
Oct 30 19:22:04.851419 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:04.851381    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:06.152838 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:06.152806    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:06.412894 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:06.401746    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:06.822285 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:06.822248    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"]
Oct 30 19:22:06.828296 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:06.828272    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"
Oct 30 19:22:06.842976 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:06.842954    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:06.891662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:06.891624    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtmqd\" (UniqueName: \"kubernetes.io/projected/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-kube-api-access-mtmqd\") pod \"payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps\" (UID: \"46f7284e-cdd6-49c8-9f65-daff0f30f8bb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"
Oct 30 19:22:06.892166 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:06.892138    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps\" (UID: \"46f7284e-cdd6-49c8-9f65-daff0f30f8bb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"
Oct 30 19:22:06.895534 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:06.895507    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"]
Oct 30 19:22:06.925567 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:06.925542    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-d6b5ac\""
Oct 30 19:22:06.925738 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:06.925593    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-d6b5ac-dockercfg-wh7jg\""
Oct 30 19:22:06.993200 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:06.993165    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps\" (UID: \"46f7284e-cdd6-49c8-9f65-daff0f30f8bb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"
Oct 30 19:22:06.993369 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:06.993288    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtmqd\" (UniqueName: \"kubernetes.io/projected/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-kube-api-access-mtmqd\") pod \"payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps\" (UID: \"46f7284e-cdd6-49c8-9f65-daff0f30f8bb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"
Oct 30 19:22:06.993886 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:06.993860    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps\" (UID: \"46f7284e-cdd6-49c8-9f65-daff0f30f8bb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"
Oct 30 19:22:07.050694 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:07.050671    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtmqd\" (UniqueName: \"kubernetes.io/projected/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-kube-api-access-mtmqd\") pod \"payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps\" (UID: \"46f7284e-cdd6-49c8-9f65-daff0f30f8bb\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"
Oct 30 19:22:07.086287 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:07.086201    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:07.141358 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:07.141334    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"
Oct 30 19:22:07.534140 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:07.534108    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"]
Oct 30 19:22:07.536214 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:22:07.536180    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46f7284e_cdd6_49c8_9f65_daff0f30f8bb.slice/crio-33a0c6f4bb61ac8e7602d43b02a8fef90aa6d7e02bfae7f2db2f1a681b34b298 WatchSource:0}: Error finding container 33a0c6f4bb61ac8e7602d43b02a8fef90aa6d7e02bfae7f2db2f1a681b34b298: Status 404 returned error can't find the container with id 33a0c6f4bb61ac8e7602d43b02a8fef90aa6d7e02bfae7f2db2f1a681b34b298
Oct 30 19:22:08.186457 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:08.186424    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps" event={"ID":"46f7284e-cdd6-49c8-9f65-daff0f30f8bb","Type":"ContainerStarted","Data":"5eca2a6f138d93f20f2d624ebe22754026175b5e646270b8b338d259c67d3c26"}
Oct 30 19:22:08.186631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:08.186474    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps" event={"ID":"46f7284e-cdd6-49c8-9f65-daff0f30f8bb","Type":"ContainerStarted","Data":"33a0c6f4bb61ac8e7602d43b02a8fef90aa6d7e02bfae7f2db2f1a681b34b298"}
Oct 30 19:22:08.186631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:08.186514    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"
Oct 30 19:22:08.297391 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:08.297345    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps" podStartSLOduration=2.297332381 podStartE2EDuration="2.297332381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:22:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:22:08.29522475 +0000 UTC m=+2101.241392654" watchObservedRunningTime="2026-10-30 19:22:08.297332381 +0000 UTC m=+2101.243500284"
Oct 30 19:22:08.553570 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:08.553538    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:08.797280 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:08.797242    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:10.167794 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:10.167766    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:22:11.681201 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:11.681164    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:16.538449 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:16.538413    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:17.111123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:17.111051    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:22:19.194809 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:19.194783    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"
Oct 30 19:22:23.041020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:23.030715    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:29.228443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:29.228404    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:33.383640 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:22:33.383604    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:22:33.387635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:33.387609    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:33.784768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:33.784732    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:33.986944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:33.986900    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:34.169692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:34.169610    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:38.746060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:38.745720    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:43.273911 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:43.273821    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:43.518731 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:43.518691    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:43.734096 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:43.733993    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:45.868863 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:45.868818    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:22:48.078351 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:48.078311    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:52.866578 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:52.866534    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:54.907527 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:54.907489    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:56.523343 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:56.523310    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"]
Oct 30 19:22:56.624160 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:56.624129    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"]
Oct 30 19:22:56.900849 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:56.900761    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:57.137239 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:57.137201    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:22:57.793707 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:22:57.793671    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:01.524570 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:01.524531    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk" podUID="90a10f6f-4267-4abe-8ad2-6d4b31456575" containerName="payload-processing" containerID="cri-o://92cc1abf0de3ac816862142f3159d877f57e2964b752be9192ae5f6060bbd624" gracePeriod=25
Oct 30 19:23:01.625399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:01.625363    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr" podUID="ce342a00-bd7c-42ca-a1ae-a686c367493c" containerName="payload-pre-processing" containerID="cri-o://5773ca92716882bba7b52d703d9726ce42f1b5a6043a0623c657fecb8f58317e" gracePeriod=25
Oct 30 19:23:01.794935 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:01.794876    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"
Oct 30 19:23:01.902462 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:01.902440    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"
Oct 30 19:23:01.973184 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:01.973160    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/90a10f6f-4267-4abe-8ad2-6d4b31456575-kube-api-access-nlmkx\" (UniqueName: \"kubernetes.io/projected/90a10f6f-4267-4abe-8ad2-6d4b31456575-kube-api-access-nlmkx\") pod \"90a10f6f-4267-4abe-8ad2-6d4b31456575\" (UID: \"90a10f6f-4267-4abe-8ad2-6d4b31456575\") "
Oct 30 19:23:01.973334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:01.973224    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/90a10f6f-4267-4abe-8ad2-6d4b31456575-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90a10f6f-4267-4abe-8ad2-6d4b31456575-plugins-config-volume\") pod \"90a10f6f-4267-4abe-8ad2-6d4b31456575\" (UID: \"90a10f6f-4267-4abe-8ad2-6d4b31456575\") "
Oct 30 19:23:01.973611 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:01.973589    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/90a10f6f-4267-4abe-8ad2-6d4b31456575-plugins-config-volume" pod "90a10f6f-4267-4abe-8ad2-6d4b31456575" (UID: "90a10f6f-4267-4abe-8ad2-6d4b31456575"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:01.975312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:01.975294    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/90a10f6f-4267-4abe-8ad2-6d4b31456575-kube-api-access-nlmkx" pod "90a10f6f-4267-4abe-8ad2-6d4b31456575" (UID: "90a10f6f-4267-4abe-8ad2-6d4b31456575"). InnerVolumeSpecName "kube-api-access-nlmkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:02.074222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.074152    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce342a00-bd7c-42ca-a1ae-a686c367493c-kube-api-access-qsz7r\" (UniqueName: \"kubernetes.io/projected/ce342a00-bd7c-42ca-a1ae-a686c367493c-kube-api-access-qsz7r\") pod \"ce342a00-bd7c-42ca-a1ae-a686c367493c\" (UID: \"ce342a00-bd7c-42ca-a1ae-a686c367493c\") "
Oct 30 19:23:02.074361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.074308    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ce342a00-bd7c-42ca-a1ae-a686c367493c-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce342a00-bd7c-42ca-a1ae-a686c367493c-plugins-config-volume\") pod \"ce342a00-bd7c-42ca-a1ae-a686c367493c\" (UID: \"ce342a00-bd7c-42ca-a1ae-a686c367493c\") "
Oct 30 19:23:02.074679 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.074657    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ce342a00-bd7c-42ca-a1ae-a686c367493c-plugins-config-volume" pod "ce342a00-bd7c-42ca-a1ae-a686c367493c" (UID: "ce342a00-bd7c-42ca-a1ae-a686c367493c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:02.074749 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.074680    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlmkx\" (UniqueName: \"kubernetes.io/projected/90a10f6f-4267-4abe-8ad2-6d4b31456575-kube-api-access-nlmkx\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:02.074749 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.074693    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/90a10f6f-4267-4abe-8ad2-6d4b31456575-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:02.076315 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.076295    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce342a00-bd7c-42ca-a1ae-a686c367493c-kube-api-access-qsz7r" pod "ce342a00-bd7c-42ca-a1ae-a686c367493c" (UID: "ce342a00-bd7c-42ca-a1ae-a686c367493c"). InnerVolumeSpecName "kube-api-access-qsz7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:02.175538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.175512    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ce342a00-bd7c-42ca-a1ae-a686c367493c-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:02.175538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.175534    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsz7r\" (UniqueName: \"kubernetes.io/projected/ce342a00-bd7c-42ca-a1ae-a686c367493c-kube-api-access-qsz7r\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:02.514621 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.514586    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:02.582173 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.582128    5704 generic.go:356] "Generic (PLEG): container finished" podID="ce342a00-bd7c-42ca-a1ae-a686c367493c" containerID="5773ca92716882bba7b52d703d9726ce42f1b5a6043a0623c657fecb8f58317e" exitCode=0
Oct 30 19:23:02.582644 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.582422    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"
Oct 30 19:23:02.582998 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.582969    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr" event={"ID":"ce342a00-bd7c-42ca-a1ae-a686c367493c","Type":"ContainerDied","Data":"5773ca92716882bba7b52d703d9726ce42f1b5a6043a0623c657fecb8f58317e"}
Oct 30 19:23:02.583122 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.583064    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr" event={"ID":"ce342a00-bd7c-42ca-a1ae-a686c367493c","Type":"ContainerDied","Data":"17f206ac9067bab4ec40d194696e4597243027bcb177f20e722eeab2b155f698"}
Oct 30 19:23:02.583122 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.583099    5704 scope.go:122] "RemoveContainer" containerID="5773ca92716882bba7b52d703d9726ce42f1b5a6043a0623c657fecb8f58317e"
Oct 30 19:23:02.590186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.589987    5704 generic.go:356] "Generic (PLEG): container finished" podID="90a10f6f-4267-4abe-8ad2-6d4b31456575" containerID="92cc1abf0de3ac816862142f3159d877f57e2964b752be9192ae5f6060bbd624" exitCode=0
Oct 30 19:23:02.590186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.590084    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk" event={"ID":"90a10f6f-4267-4abe-8ad2-6d4b31456575","Type":"ContainerDied","Data":"92cc1abf0de3ac816862142f3159d877f57e2964b752be9192ae5f6060bbd624"}
Oct 30 19:23:02.590186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.590122    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"
Oct 30 19:23:02.590186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.590140    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk" event={"ID":"90a10f6f-4267-4abe-8ad2-6d4b31456575","Type":"ContainerDied","Data":"f647d37cb2147ab6c3505d34c98ed1948c0c2cb3dca51878324be4493a67da70"}
Oct 30 19:23:02.598674 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.598659    5704 scope.go:122] "RemoveContainer" containerID="5773ca92716882bba7b52d703d9726ce42f1b5a6043a0623c657fecb8f58317e"
Oct 30 19:23:02.598935 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:23:02.598914    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5773ca92716882bba7b52d703d9726ce42f1b5a6043a0623c657fecb8f58317e\": container with ID starting with 5773ca92716882bba7b52d703d9726ce42f1b5a6043a0623c657fecb8f58317e not found: ID does not exist" containerID="5773ca92716882bba7b52d703d9726ce42f1b5a6043a0623c657fecb8f58317e"
Oct 30 19:23:02.598997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.598941    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5773ca92716882bba7b52d703d9726ce42f1b5a6043a0623c657fecb8f58317e"} err="failed to get container status \"5773ca92716882bba7b52d703d9726ce42f1b5a6043a0623c657fecb8f58317e\": rpc error: code = NotFound desc = could not find container \"5773ca92716882bba7b52d703d9726ce42f1b5a6043a0623c657fecb8f58317e\": container with ID starting with 5773ca92716882bba7b52d703d9726ce42f1b5a6043a0623c657fecb8f58317e not found: ID does not exist"
Oct 30 19:23:02.598997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.598957    5704 scope.go:122] "RemoveContainer" containerID="92cc1abf0de3ac816862142f3159d877f57e2964b752be9192ae5f6060bbd624"
Oct 30 19:23:02.607998 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.607980    5704 scope.go:122] "RemoveContainer" containerID="92cc1abf0de3ac816862142f3159d877f57e2964b752be9192ae5f6060bbd624"
Oct 30 19:23:02.608263 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:23:02.608243    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92cc1abf0de3ac816862142f3159d877f57e2964b752be9192ae5f6060bbd624\": container with ID starting with 92cc1abf0de3ac816862142f3159d877f57e2964b752be9192ae5f6060bbd624 not found: ID does not exist" containerID="92cc1abf0de3ac816862142f3159d877f57e2964b752be9192ae5f6060bbd624"
Oct 30 19:23:02.608306 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.608271    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92cc1abf0de3ac816862142f3159d877f57e2964b752be9192ae5f6060bbd624"} err="failed to get container status \"92cc1abf0de3ac816862142f3159d877f57e2964b752be9192ae5f6060bbd624\": rpc error: code = NotFound desc = could not find container \"92cc1abf0de3ac816862142f3159d877f57e2964b752be9192ae5f6060bbd624\": container with ID starting with 92cc1abf0de3ac816862142f3159d877f57e2964b752be9192ae5f6060bbd624 not found: ID does not exist"
Oct 30 19:23:02.747653 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.747628    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"]
Oct 30 19:23:02.751686 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.751666    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-65d741-b4544bbdd-gkxvk"]
Oct 30 19:23:02.895657 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.895598    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"]
Oct 30 19:23:02.900346 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:02.900324    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-65d741-86b685c57f-pwjwr"]
Oct 30 19:23:03.187510 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:03.187430    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="90a10f6f-4267-4abe-8ad2-6d4b31456575" path="/var/lib/kubelet/pods/90a10f6f-4267-4abe-8ad2-6d4b31456575/volumes"
Oct 30 19:23:03.187920 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:03.187900    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce342a00-bd7c-42ca-a1ae-a686c367493c" path="/var/lib/kubelet/pods/ce342a00-bd7c-42ca-a1ae-a686c367493c/volumes"
Oct 30 19:23:03.382958 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:23:03.382924    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:23:06.485800 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:06.485769    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:07.335058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:07.334585    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:10.717693 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:10.717657    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:13.626997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.626965    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"]
Oct 30 19:23:13.627687 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.627669    5704 cpu_manager.go:414] "containerMap: removing container" podUID="ce342a00-bd7c-42ca-a1ae-a686c367493c" containerName="payload-pre-processing"
Oct 30 19:23:13.627732 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.627689    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce342a00-bd7c-42ca-a1ae-a686c367493c" containerName="payload-pre-processing"
Oct 30 19:23:13.627732 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.627712    5704 cpu_manager.go:414] "containerMap: removing container" podUID="90a10f6f-4267-4abe-8ad2-6d4b31456575" containerName="payload-processing"
Oct 30 19:23:13.627732 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.627717    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="90a10f6f-4267-4abe-8ad2-6d4b31456575" containerName="payload-processing"
Oct 30 19:23:13.627827 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.627798    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="90a10f6f-4267-4abe-8ad2-6d4b31456575" containerName="payload-processing"
Oct 30 19:23:13.627827 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.627815    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce342a00-bd7c-42ca-a1ae-a686c367493c" containerName="payload-pre-processing"
Oct 30 19:23:13.632374 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.632355    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:13.681259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.681234    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-d6b5ac-serving-cert\""
Oct 30 19:23:13.691556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.691533    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"]
Oct 30 19:23:13.715172 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.715134    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9gvq\" (UniqueName: \"kubernetes.io/projected/a7704518-d998-47b2-8044-f87a597ce053-kube-api-access-x9gvq\") pod \"maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:13.715334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.715201    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-metrics-certs\") pod \"maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:13.715334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.715286    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-maas-api-tls\") pod \"maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:13.715465 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.715366    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7704518-d998-47b2-8044-f87a597ce053-openshift-service-ca\") pod \"maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:13.816508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.816476    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7704518-d998-47b2-8044-f87a597ce053-openshift-service-ca\") pod \"maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:13.816661 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.816585    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9gvq\" (UniqueName: \"kubernetes.io/projected/a7704518-d998-47b2-8044-f87a597ce053-kube-api-access-x9gvq\") pod \"maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:13.816661 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.816633    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-metrics-certs\") pod \"maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:13.816760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.816667    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-maas-api-tls\") pod \"maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:13.816830 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:23:13.816809    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-serving-cert: secret "maas-api-e2e-api-b-d6b5ac-serving-cert" not found
Oct 30 19:23:13.816896 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:23:13.816886    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-maas-api-tls podName:a7704518-d998-47b2-8044-f87a597ce053 nodeName:}" failed. No retries permitted until 2026-10-30 19:23:14.316870365 +0000 UTC m=+2167.263038263 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-maas-api-tls") pod "maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v" (UID: "a7704518-d998-47b2-8044-f87a597ce053") : secret "maas-api-e2e-api-b-d6b5ac-serving-cert" not found
Oct 30 19:23:13.817229 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.817204    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7704518-d998-47b2-8044-f87a597ce053-openshift-service-ca\") pod \"maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:13.819149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.819127    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-metrics-certs\") pod \"maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:13.858863 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.858816    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:13.926108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:13.911737    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9gvq\" (UniqueName: \"kubernetes.io/projected/a7704518-d998-47b2-8044-f87a597ce053-kube-api-access-x9gvq\") pod \"maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:14.091191 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:14.091152    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:14.337552 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:14.337520    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-maas-api-tls\") pod \"maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:14.339749 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:14.339731    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-maas-api-tls\") pod \"maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:14.545346 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:14.545314    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:14.917109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:14.916993    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"]
Oct 30 19:23:14.920543 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:23:14.920517    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7704518_d998_47b2_8044_f87a597ce053.slice/crio-753b608d4bbe96b8425c4b6cd9bb56e66b28617976c19955725a605a996b4241 WatchSource:0}: Error finding container 753b608d4bbe96b8425c4b6cd9bb56e66b28617976c19955725a605a996b4241: Status 404 returned error can't find the container with id 753b608d4bbe96b8425c4b6cd9bb56e66b28617976c19955725a605a996b4241
Oct 30 19:23:15.272875 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:15.272845    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:15.510006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:15.509977    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:15.679834 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:15.679758    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v" event={"ID":"a7704518-d998-47b2-8044-f87a597ce053","Type":"ContainerStarted","Data":"753b608d4bbe96b8425c4b6cd9bb56e66b28617976c19955725a605a996b4241"}
Oct 30 19:23:16.688287 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:16.688248    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v" event={"ID":"a7704518-d998-47b2-8044-f87a597ce053","Type":"ContainerStarted","Data":"d5636554ad3b042af9deae1d3f7989775e97a56ae52ed069a5b880d522e95879"}
Oct 30 19:23:16.688789 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:16.688465    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:16.831994 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:16.831940    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v" podStartSLOduration=2.983639566 podStartE2EDuration="3.831926745s" totalImagesPullingTime="848.287179ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:23:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:23:16.829389367 +0000 UTC m=+2169.775557297" watchObservedRunningTime="2026-10-30 19:23:16.831926745 +0000 UTC m=+2169.778094648"
Oct 30 19:23:18.909764 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:18.909491    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:22.559356 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:22.559320    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"]
Oct 30 19:23:22.559853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:22.559593    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerName="istio-proxy" containerID="cri-o://72e25eec0ef3a4f3f9f6e684854fe0d7b4ede1038b6866212223ed1d02d18edd" gracePeriod=30
Oct 30 19:23:22.605438 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:22.605404    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"]
Oct 30 19:23:22.605649 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:22.605628    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerName="istio-proxy" containerID="cri-o://3b36a5774c209783b5ce964d2885f1b0f11264897eacc6220c84cc089f236ee5" gracePeriod=30
Oct 30 19:23:22.607995 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:22.607968    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"]
Oct 30 19:23:22.608210 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:22.608183    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" podUID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" containerName="istio-proxy" containerID="cri-o://73758a707786321b024ae500946a263af46baa792ba58aa6e98568e2e5e2c931" gracePeriod=30
Oct 30 19:23:22.700674 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:22.700647    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:23:22.895913 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:22.890261    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:23.607416 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:23.607364    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:24.331723 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:24.331688    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:23:24.331912 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:24.331748    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" podUID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:23:24.505735 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:24.505679    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:23:24.505919 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:24.505738    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:23:24.548660 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:24.548627    5704 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:23:24.548782 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:24.548683    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:23:27.779480 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:27.779445    5704 generic.go:356] "Generic (PLEG): container finished" podID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerID="72e25eec0ef3a4f3f9f6e684854fe0d7b4ede1038b6866212223ed1d02d18edd" exitCode=0
Oct 30 19:23:27.782161 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:27.779556    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" event={"ID":"d494fbe9-42fe-4523-b7c2-a6b1771c18e3","Type":"ContainerDied","Data":"72e25eec0ef3a4f3f9f6e684854fe0d7b4ede1038b6866212223ed1d02d18edd"}
Oct 30 19:23:27.785777 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:27.785746    5704 generic.go:356] "Generic (PLEG): container finished" podID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" containerID="73758a707786321b024ae500946a263af46baa792ba58aa6e98568e2e5e2c931" exitCode=0
Oct 30 19:23:27.785894 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:27.785877    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" event={"ID":"b0b50b44-9222-4120-ab91-f11dfd78f7e5","Type":"ContainerDied","Data":"73758a707786321b024ae500946a263af46baa792ba58aa6e98568e2e5e2c931"}
Oct 30 19:23:27.790813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:27.790774    5704 generic.go:356] "Generic (PLEG): container finished" podID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerID="3b36a5774c209783b5ce964d2885f1b0f11264897eacc6220c84cc089f236ee5" exitCode=0
Oct 30 19:23:27.791055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:27.791027    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" event={"ID":"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2","Type":"ContainerDied","Data":"3b36a5774c209783b5ce964d2885f1b0f11264897eacc6220c84cc089f236ee5"}
Oct 30 19:23:27.898864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:27.898835    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:23:27.954908 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:27.954882    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:23:27.958573 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:27.958554    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:23:28.021834 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.021807    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-token\" (UniqueName: \"kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-token\") pod \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") "
Oct 30 19:23:28.022008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.021854    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-data\") pod \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") "
Oct 30 19:23:28.022008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.021970    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istiod-ca-cert\") pod \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") "
Oct 30 19:23:28.022156 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022017    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istiod-ca-cert\") pod \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") "
Oct 30 19:23:28.022156 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022095    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-credential-socket\") pod \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") "
Oct 30 19:23:28.022156 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022120    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-certs\") pod \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") "
Oct 30 19:23:28.022156 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022137    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-podinfo\") pod \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") "
Oct 30 19:23:28.022156 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022147    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-data" pod "d494fbe9-42fe-4523-b7c2-a6b1771c18e3" (UID: "d494fbe9-42fe-4523-b7c2-a6b1771c18e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.022422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022167    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-certs\") pod \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") "
Oct 30 19:23:28.022422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022202    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-podinfo\") pod \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") "
Oct 30 19:23:28.022422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022238    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istiod-ca-cert\") pod \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") "
Oct 30 19:23:28.022422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022272    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-kube-api-access-bwnwg\" (UniqueName: \"kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-kube-api-access-bwnwg\") pod \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") "
Oct 30 19:23:28.022422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022309    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-envoy\") pod \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") "
Oct 30 19:23:28.022422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022352    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-credential-socket\") pod \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") "
Oct 30 19:23:28.022422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022390    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-socket\") pod \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") "
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022430    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-socket\") pod \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") "
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022441    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-certs" pod "b0b50b44-9222-4120-ab91-f11dfd78f7e5" (UID: "b0b50b44-9222-4120-ab91-f11dfd78f7e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022460    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-data\") pod \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") "
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022467    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istiod-ca-cert" pod "d494fbe9-42fe-4523-b7c2-a6b1771c18e3" (UID: "d494fbe9-42fe-4523-b7c2-a6b1771c18e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022493    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istiod-ca-cert" pod "b0b50b44-9222-4120-ab91-f11dfd78f7e5" (UID: "b0b50b44-9222-4120-ab91-f11dfd78f7e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022512    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-data\") pod \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") "
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022551    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-socket\") pod \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") "
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022583    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-envoy\") pod \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") "
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022611    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-kube-api-access-nz4bq\" (UniqueName: \"kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-kube-api-access-nz4bq\") pod \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") "
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022702    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-credential-socket" pod "d494fbe9-42fe-4523-b7c2-a6b1771c18e3" (UID: "d494fbe9-42fe-4523-b7c2-a6b1771c18e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022711    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-credential-socket" pod "b0b50b44-9222-4120-ab91-f11dfd78f7e5" (UID: "b0b50b44-9222-4120-ab91-f11dfd78f7e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022729    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-kube-api-access-rrjsp\" (UniqueName: \"kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-kube-api-access-rrjsp\") pod \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") "
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022799    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-podinfo\") pod \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") "
Oct 30 19:23:28.022820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022807    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-socket" pod "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" (UID: "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.023586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022914    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-certs\") pod \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") "
Oct 30 19:23:28.023586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.022974    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-credential-socket\") pod \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\" (UID: \"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2\") "
Oct 30 19:23:28.023586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023014    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-envoy\") pod \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") "
Oct 30 19:23:28.023586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023047    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istiod-ca-cert" pod "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" (UID: "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:28.023586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023077    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-token\" (UniqueName: \"kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-token\") pod \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\" (UID: \"d494fbe9-42fe-4523-b7c2-a6b1771c18e3\") "
Oct 30 19:23:28.023586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023120    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-token\" (UniqueName: \"kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-token\") pod \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\" (UID: \"b0b50b44-9222-4120-ab91-f11dfd78f7e5\") "
Oct 30 19:23:28.023586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023291    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-certs" pod "d494fbe9-42fe-4523-b7c2-a6b1771c18e3" (UID: "d494fbe9-42fe-4523-b7c2-a6b1771c18e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.023955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023784    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.023955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023806    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.023955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023820    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.023955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023819    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-certs" pod "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" (UID: "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.023955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023835    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.023955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023842    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-credential-socket" pod "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" (UID: "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.023955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023850    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.023955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023886    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.023955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023906    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.023955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023920    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.023955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.023934    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.024989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.024146    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-data" pod "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" (UID: "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.024989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.024348    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-socket" pod "d494fbe9-42fe-4523-b7c2-a6b1771c18e3" (UID: "d494fbe9-42fe-4523-b7c2-a6b1771c18e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.024989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.024393    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-socket" pod "b0b50b44-9222-4120-ab91-f11dfd78f7e5" (UID: "b0b50b44-9222-4120-ab91-f11dfd78f7e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.024989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.024569    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-data" pod "b0b50b44-9222-4120-ab91-f11dfd78f7e5" (UID: "b0b50b44-9222-4120-ab91-f11dfd78f7e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.025405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.025376    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-kube-api-access-bwnwg" pod "b0b50b44-9222-4120-ab91-f11dfd78f7e5" (UID: "b0b50b44-9222-4120-ab91-f11dfd78f7e5"). InnerVolumeSpecName "kube-api-access-bwnwg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:28.025489 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.025437    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-envoy" pod "d494fbe9-42fe-4523-b7c2-a6b1771c18e3" (UID: "d494fbe9-42fe-4523-b7c2-a6b1771c18e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.025782 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.025754    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-podinfo" pod "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" (UID: "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:23:28.025999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.025970    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-podinfo" pod "d494fbe9-42fe-4523-b7c2-a6b1771c18e3" (UID: "d494fbe9-42fe-4523-b7c2-a6b1771c18e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:23:28.026284 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.026249    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-envoy" pod "b0b50b44-9222-4120-ab91-f11dfd78f7e5" (UID: "b0b50b44-9222-4120-ab91-f11dfd78f7e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.026544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.026512    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-podinfo" pod "b0b50b44-9222-4120-ab91-f11dfd78f7e5" (UID: "b0b50b44-9222-4120-ab91-f11dfd78f7e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:23:28.026544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.026521    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-token" pod "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" (UID: "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:28.026724 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.026665    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-kube-api-access-rrjsp" pod "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" (UID: "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2"). InnerVolumeSpecName "kube-api-access-rrjsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:28.027121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.027101    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-kube-api-access-nz4bq" pod "d494fbe9-42fe-4523-b7c2-a6b1771c18e3" (UID: "d494fbe9-42fe-4523-b7c2-a6b1771c18e3"). InnerVolumeSpecName "kube-api-access-nz4bq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:28.027203 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.027119    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-envoy" pod "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" (UID: "3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:23:28.027393 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.027380    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-token" pod "b0b50b44-9222-4120-ab91-f11dfd78f7e5" (UID: "b0b50b44-9222-4120-ab91-f11dfd78f7e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:28.027556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.027538    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-token" pod "d494fbe9-42fe-4523-b7c2-a6b1771c18e3" (UID: "d494fbe9-42fe-4523-b7c2-a6b1771c18e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:28.124724 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124645    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrjsp\" (UniqueName: \"kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-kube-api-access-rrjsp\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124724 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124693    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124724 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124704    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124724 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124713    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124724 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124721    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124730    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124738    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124746    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124755    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124763    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124771    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwnwg\" (UniqueName: \"kubernetes.io/projected/b0b50b44-9222-4120-ab91-f11dfd78f7e5-kube-api-access-bwnwg\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124780    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124788    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124796    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124805    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124812    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0b50b44-9222-4120-ab91-f11dfd78f7e5-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124819    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.124999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.124827    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nz4bq\" (UniqueName: \"kubernetes.io/projected/d494fbe9-42fe-4523-b7c2-a6b1771c18e3-kube-api-access-nz4bq\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:28.400008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.399924    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"]
Oct 30 19:23:28.459243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.459211    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"]
Oct 30 19:23:28.668694 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.668617    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:28.799382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.799355    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"
Oct 30 19:23:28.799819 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.799382    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t" event={"ID":"d494fbe9-42fe-4523-b7c2-a6b1771c18e3","Type":"ContainerDied","Data":"c69badfb763e4d79c54720cf4d7b4810bb1022c5a5625cf990613964263e914d"}
Oct 30 19:23:28.799819 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.799450    5704 scope.go:122] "RemoveContainer" containerID="72e25eec0ef3a4f3f9f6e684854fe0d7b4ede1038b6866212223ed1d02d18edd"
Oct 30 19:23:28.800988 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.800972    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"
Oct 30 19:23:28.801107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.800983    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h" event={"ID":"b0b50b44-9222-4120-ab91-f11dfd78f7e5","Type":"ContainerDied","Data":"21fa679b87af6d62f3832eb97d0691bf9515360b35c571085948c898452ae929"}
Oct 30 19:23:28.802553 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.802528    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt" event={"ID":"3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2","Type":"ContainerDied","Data":"8bf89134e2e60d08e80e860ab18160f80011e40a03a95908939e28de1bb10fd6"}
Oct 30 19:23:28.802653 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.802551    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"
Oct 30 19:23:28.812804 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.812782    5704 scope.go:122] "RemoveContainer" containerID="73758a707786321b024ae500946a263af46baa792ba58aa6e98568e2e5e2c931"
Oct 30 19:23:28.822993 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.822977    5704 scope.go:122] "RemoveContainer" containerID="3b36a5774c209783b5ce964d2885f1b0f11264897eacc6220c84cc089f236ee5"
Oct 30 19:23:28.898078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.898021    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:28.958059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.957962    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"]
Oct 30 19:23:28.961312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:28.961287    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-6mr4h"]
Oct 30 19:23:29.063858 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:29.063831    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"]
Oct 30 19:23:29.068688 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:29.068661    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-4kdgt"]
Oct 30 19:23:29.170972 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:29.170949    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"]
Oct 30 19:23:29.183263 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:29.183235    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" path="/var/lib/kubelet/pods/3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2/volumes"
Oct 30 19:23:29.183733 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:29.183719    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" path="/var/lib/kubelet/pods/b0b50b44-9222-4120-ab91-f11dfd78f7e5/volumes"
Oct 30 19:23:29.184014 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:29.184002    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-ccr5t"]
Oct 30 19:23:31.180477 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:31.180445    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" path="/var/lib/kubelet/pods/d494fbe9-42fe-4523-b7c2-a6b1771c18e3/volumes"
Oct 30 19:23:32.380819 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:32.380782    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:23:33.384177 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:23:33.384137    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:23:33.400513 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.400482    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx" podUID="af419107-fbad-4021-88a0-aef9c988537e" containerName="payload-processing" containerID="cri-o://0c2a5c7cafec01804dacd58e21fcff9d20b3fced08e2769aa195b3e24562cf3c" gracePeriod=25
Oct 30 19:23:33.460505 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.460468    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d" podUID="6d681bc0-72b9-4624-8978-48afb725d311" containerName="payload-pre-processing" containerID="cri-o://b7311c6314393d7d385bc51f4026f5106cb9c6d13f1da0860da269e26ca5448a" gracePeriod=25
Oct 30 19:23:33.639045 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.638821    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:33.767055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.767003    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"
Oct 30 19:23:33.809784 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.809761    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"
Oct 30 19:23:33.849520 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.849496    5704 generic.go:356] "Generic (PLEG): container finished" podID="af419107-fbad-4021-88a0-aef9c988537e" containerID="0c2a5c7cafec01804dacd58e21fcff9d20b3fced08e2769aa195b3e24562cf3c" exitCode=0
Oct 30 19:23:33.849675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.849546    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"
Oct 30 19:23:33.849675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.849572    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx" event={"ID":"af419107-fbad-4021-88a0-aef9c988537e","Type":"ContainerDied","Data":"0c2a5c7cafec01804dacd58e21fcff9d20b3fced08e2769aa195b3e24562cf3c"}
Oct 30 19:23:33.849675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.849620    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx" event={"ID":"af419107-fbad-4021-88a0-aef9c988537e","Type":"ContainerDied","Data":"096ae654dc53c4a552e501e5b2a5edff7496c0db1cfe8f25d3fb0f1ccd87736c"}
Oct 30 19:23:33.849675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.849642    5704 scope.go:122] "RemoveContainer" containerID="0c2a5c7cafec01804dacd58e21fcff9d20b3fced08e2769aa195b3e24562cf3c"
Oct 30 19:23:33.850968 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.850946    5704 generic.go:356] "Generic (PLEG): container finished" podID="6d681bc0-72b9-4624-8978-48afb725d311" containerID="b7311c6314393d7d385bc51f4026f5106cb9c6d13f1da0860da269e26ca5448a" exitCode=0
Oct 30 19:23:33.851092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.850991    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"
Oct 30 19:23:33.851092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.851013    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d" event={"ID":"6d681bc0-72b9-4624-8978-48afb725d311","Type":"ContainerDied","Data":"b7311c6314393d7d385bc51f4026f5106cb9c6d13f1da0860da269e26ca5448a"}
Oct 30 19:23:33.851187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.851093    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d" event={"ID":"6d681bc0-72b9-4624-8978-48afb725d311","Type":"ContainerDied","Data":"7803c625db7a5ed155dc51fcfe3847703c4ee0c05900501642bc6c42ff823d79"}
Oct 30 19:23:33.859604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.859590    5704 scope.go:122] "RemoveContainer" containerID="0c2a5c7cafec01804dacd58e21fcff9d20b3fced08e2769aa195b3e24562cf3c"
Oct 30 19:23:33.859863 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:23:33.859842    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c2a5c7cafec01804dacd58e21fcff9d20b3fced08e2769aa195b3e24562cf3c\": container with ID starting with 0c2a5c7cafec01804dacd58e21fcff9d20b3fced08e2769aa195b3e24562cf3c not found: ID does not exist" containerID="0c2a5c7cafec01804dacd58e21fcff9d20b3fced08e2769aa195b3e24562cf3c"
Oct 30 19:23:33.859925 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.859869    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c2a5c7cafec01804dacd58e21fcff9d20b3fced08e2769aa195b3e24562cf3c"} err="failed to get container status \"0c2a5c7cafec01804dacd58e21fcff9d20b3fced08e2769aa195b3e24562cf3c\": rpc error: code = NotFound desc = could not find container \"0c2a5c7cafec01804dacd58e21fcff9d20b3fced08e2769aa195b3e24562cf3c\": container with ID starting with 0c2a5c7cafec01804dacd58e21fcff9d20b3fced08e2769aa195b3e24562cf3c not found: ID does not exist"
Oct 30 19:23:33.859925 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.859886    5704 scope.go:122] "RemoveContainer" containerID="b7311c6314393d7d385bc51f4026f5106cb9c6d13f1da0860da269e26ca5448a"
Oct 30 19:23:33.868626 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.868601    5704 scope.go:122] "RemoveContainer" containerID="b7311c6314393d7d385bc51f4026f5106cb9c6d13f1da0860da269e26ca5448a"
Oct 30 19:23:33.868890 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:23:33.868868    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7311c6314393d7d385bc51f4026f5106cb9c6d13f1da0860da269e26ca5448a\": container with ID starting with b7311c6314393d7d385bc51f4026f5106cb9c6d13f1da0860da269e26ca5448a not found: ID does not exist" containerID="b7311c6314393d7d385bc51f4026f5106cb9c6d13f1da0860da269e26ca5448a"
Oct 30 19:23:33.868949 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.868899    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7311c6314393d7d385bc51f4026f5106cb9c6d13f1da0860da269e26ca5448a"} err="failed to get container status \"b7311c6314393d7d385bc51f4026f5106cb9c6d13f1da0860da269e26ca5448a\": rpc error: code = NotFound desc = could not find container \"b7311c6314393d7d385bc51f4026f5106cb9c6d13f1da0860da269e26ca5448a\": container with ID starting with b7311c6314393d7d385bc51f4026f5106cb9c6d13f1da0860da269e26ca5448a not found: ID does not exist"
Oct 30 19:23:33.912647 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.912581    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af419107-fbad-4021-88a0-aef9c988537e-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af419107-fbad-4021-88a0-aef9c988537e-plugins-config-volume\") pod \"af419107-fbad-4021-88a0-aef9c988537e\" (UID: \"af419107-fbad-4021-88a0-aef9c988537e\") "
Oct 30 19:23:33.912647 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.912620    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af419107-fbad-4021-88a0-aef9c988537e-kube-api-access-jx46t\" (UniqueName: \"kubernetes.io/projected/af419107-fbad-4021-88a0-aef9c988537e-kube-api-access-jx46t\") pod \"af419107-fbad-4021-88a0-aef9c988537e\" (UID: \"af419107-fbad-4021-88a0-aef9c988537e\") "
Oct 30 19:23:33.912868 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.912667    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6d681bc0-72b9-4624-8978-48afb725d311-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d681bc0-72b9-4624-8978-48afb725d311-plugins-config-volume\") pod \"6d681bc0-72b9-4624-8978-48afb725d311\" (UID: \"6d681bc0-72b9-4624-8978-48afb725d311\") "
Oct 30 19:23:33.912868 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.912729    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6d681bc0-72b9-4624-8978-48afb725d311-kube-api-access-tbljp\" (UniqueName: \"kubernetes.io/projected/6d681bc0-72b9-4624-8978-48afb725d311-kube-api-access-tbljp\") pod \"6d681bc0-72b9-4624-8978-48afb725d311\" (UID: \"6d681bc0-72b9-4624-8978-48afb725d311\") "
Oct 30 19:23:33.913008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.912986    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af419107-fbad-4021-88a0-aef9c988537e-plugins-config-volume" pod "af419107-fbad-4021-88a0-aef9c988537e" (UID: "af419107-fbad-4021-88a0-aef9c988537e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:33.913135 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.913104    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d681bc0-72b9-4624-8978-48afb725d311-plugins-config-volume" pod "6d681bc0-72b9-4624-8978-48afb725d311" (UID: "6d681bc0-72b9-4624-8978-48afb725d311"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:33.913317 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.913292    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/af419107-fbad-4021-88a0-aef9c988537e-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:33.913422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.913319    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d681bc0-72b9-4624-8978-48afb725d311-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:33.914841 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.914813    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af419107-fbad-4021-88a0-aef9c988537e-kube-api-access-jx46t" pod "af419107-fbad-4021-88a0-aef9c988537e" (UID: "af419107-fbad-4021-88a0-aef9c988537e"). InnerVolumeSpecName "kube-api-access-jx46t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:33.914979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:33.914963    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d681bc0-72b9-4624-8978-48afb725d311-kube-api-access-tbljp" pod "6d681bc0-72b9-4624-8978-48afb725d311" (UID: "6d681bc0-72b9-4624-8978-48afb725d311"). InnerVolumeSpecName "kube-api-access-tbljp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:34.014291 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:34.014267    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jx46t\" (UniqueName: \"kubernetes.io/projected/af419107-fbad-4021-88a0-aef9c988537e-kube-api-access-jx46t\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:34.014291 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:34.014291    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbljp\" (UniqueName: \"kubernetes.io/projected/6d681bc0-72b9-4624-8978-48afb725d311-kube-api-access-tbljp\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:34.265965 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:34.265926    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"]
Oct 30 19:23:34.270431 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:34.270407    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-7a790b-699dfb97d4-qc7hx"]
Oct 30 19:23:34.371948 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:34.371920    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"]
Oct 30 19:23:34.377149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:34.377125    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-7a790b-8585986f5f-hpv7d"]
Oct 30 19:23:35.179212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:35.179181    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6d681bc0-72b9-4624-8978-48afb725d311" path="/var/lib/kubelet/pods/6d681bc0-72b9-4624-8978-48afb725d311/volumes"
Oct 30 19:23:35.179607 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:35.179592    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af419107-fbad-4021-88a0-aef9c988537e" path="/var/lib/kubelet/pods/af419107-fbad-4021-88a0-aef9c988537e/volumes"
Oct 30 19:23:39.101959 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.101930    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"]
Oct 30 19:23:39.102444 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.102123    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj" podUID="79d9bcef-ed50-4e58-9a38-ab11f215ada6" containerName="maas-api" containerID="cri-o://119463e8fdf44c842b43b9466d71458e5c3ced5428b08d265ecd7e7272d79257" gracePeriod=30
Oct 30 19:23:39.232405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.232361    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:39.399971 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.399941    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:23:39.473278 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.473252    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/79d9bcef-ed50-4e58-9a38-ab11f215ada6-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79d9bcef-ed50-4e58-9a38-ab11f215ada6-openshift-service-ca\") pod \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") "
Oct 30 19:23:39.473390 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.473360    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-metrics-certs\" (UniqueName: \"kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-metrics-certs\") pod \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") "
Oct 30 19:23:39.473440 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.473398    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/79d9bcef-ed50-4e58-9a38-ab11f215ada6-kube-api-access-zbzt8\" (UniqueName: \"kubernetes.io/projected/79d9bcef-ed50-4e58-9a38-ab11f215ada6-kube-api-access-zbzt8\") pod \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") "
Oct 30 19:23:39.473545 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.473529    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-maas-api-tls\") pod \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\" (UID: \"79d9bcef-ed50-4e58-9a38-ab11f215ada6\") "
Oct 30 19:23:39.473603 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.473580    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79d9bcef-ed50-4e58-9a38-ab11f215ada6-openshift-service-ca" pod "79d9bcef-ed50-4e58-9a38-ab11f215ada6" (UID: "79d9bcef-ed50-4e58-9a38-ab11f215ada6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:39.473988 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.473969    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/79d9bcef-ed50-4e58-9a38-ab11f215ada6-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:39.475294 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.475271    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-metrics-certs" pod "79d9bcef-ed50-4e58-9a38-ab11f215ada6" (UID: "79d9bcef-ed50-4e58-9a38-ab11f215ada6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:23:39.475444 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.475425    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-maas-api-tls" pod "79d9bcef-ed50-4e58-9a38-ab11f215ada6" (UID: "79d9bcef-ed50-4e58-9a38-ab11f215ada6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:23:39.475517 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.475431    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79d9bcef-ed50-4e58-9a38-ab11f215ada6-kube-api-access-zbzt8" pod "79d9bcef-ed50-4e58-9a38-ab11f215ada6" (UID: "79d9bcef-ed50-4e58-9a38-ab11f215ada6"). InnerVolumeSpecName "kube-api-access-zbzt8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:39.533507 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.533482    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"]
Oct 30 19:23:39.533705 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.533682    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj" podUID="60815cd9-d702-4e39-a59b-763096934ca6" containerName="payload-processing" containerID="cri-o://144c673dd6ae48c606d5493dea96f3fe4c2d64d83e8b721e4cca50a54d6515ea" gracePeriod=30
Oct 30 19:23:39.574909 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.574888    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbzt8\" (UniqueName: \"kubernetes.io/projected/79d9bcef-ed50-4e58-9a38-ab11f215ada6-kube-api-access-zbzt8\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:39.575019 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.574910    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:39.575019 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.574924    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/79d9bcef-ed50-4e58-9a38-ab11f215ada6-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:39.723100 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.723066    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:39.818576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.818551    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"
Oct 30 19:23:39.879117 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.879091    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/60815cd9-d702-4e39-a59b-763096934ca6-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60815cd9-d702-4e39-a59b-763096934ca6-plugins-config-volume\") pod \"60815cd9-d702-4e39-a59b-763096934ca6\" (UID: \"60815cd9-d702-4e39-a59b-763096934ca6\") "
Oct 30 19:23:39.879252 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.879199    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/60815cd9-d702-4e39-a59b-763096934ca6-kube-api-access-vxzvf\" (UniqueName: \"kubernetes.io/projected/60815cd9-d702-4e39-a59b-763096934ca6-kube-api-access-vxzvf\") pod \"60815cd9-d702-4e39-a59b-763096934ca6\" (UID: \"60815cd9-d702-4e39-a59b-763096934ca6\") "
Oct 30 19:23:39.879471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.879450    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60815cd9-d702-4e39-a59b-763096934ca6-plugins-config-volume" pod "60815cd9-d702-4e39-a59b-763096934ca6" (UID: "60815cd9-d702-4e39-a59b-763096934ca6"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:23:39.879606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.879593    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/60815cd9-d702-4e39-a59b-763096934ca6-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:39.881340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.881320    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60815cd9-d702-4e39-a59b-763096934ca6-kube-api-access-vxzvf" pod "60815cd9-d702-4e39-a59b-763096934ca6" (UID: "60815cd9-d702-4e39-a59b-763096934ca6"). InnerVolumeSpecName "kube-api-access-vxzvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:23:39.891438 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.891409    5704 generic.go:356] "Generic (PLEG): container finished" podID="79d9bcef-ed50-4e58-9a38-ab11f215ada6" containerID="119463e8fdf44c842b43b9466d71458e5c3ced5428b08d265ecd7e7272d79257" exitCode=0
Oct 30 19:23:39.891558 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.891473    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"
Oct 30 19:23:39.891558 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.891499    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj" event={"ID":"79d9bcef-ed50-4e58-9a38-ab11f215ada6","Type":"ContainerDied","Data":"119463e8fdf44c842b43b9466d71458e5c3ced5428b08d265ecd7e7272d79257"}
Oct 30 19:23:39.891687 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.891563    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj" event={"ID":"79d9bcef-ed50-4e58-9a38-ab11f215ada6","Type":"ContainerDied","Data":"ad55222f15496587060562cf9766202d71dada89bdf1f97c867d23f6d24b4116"}
Oct 30 19:23:39.891687 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.891594    5704 scope.go:122] "RemoveContainer" containerID="119463e8fdf44c842b43b9466d71458e5c3ced5428b08d265ecd7e7272d79257"
Oct 30 19:23:39.893429 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.893406    5704 generic.go:356] "Generic (PLEG): container finished" podID="60815cd9-d702-4e39-a59b-763096934ca6" containerID="144c673dd6ae48c606d5493dea96f3fe4c2d64d83e8b721e4cca50a54d6515ea" exitCode=0
Oct 30 19:23:39.893544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.893452    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"
Oct 30 19:23:39.893544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.893511    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj" event={"ID":"60815cd9-d702-4e39-a59b-763096934ca6","Type":"ContainerDied","Data":"144c673dd6ae48c606d5493dea96f3fe4c2d64d83e8b721e4cca50a54d6515ea"}
Oct 30 19:23:39.893636 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.893562    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj" event={"ID":"60815cd9-d702-4e39-a59b-763096934ca6","Type":"ContainerDied","Data":"037b91811b5fef11d4f702fbbc4fca2dff6feb1cd65cc58d796257befbdd70c4"}
Oct 30 19:23:39.904760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.904536    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:39.913966 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.913768    5704 scope.go:122] "RemoveContainer" containerID="119463e8fdf44c842b43b9466d71458e5c3ced5428b08d265ecd7e7272d79257"
Oct 30 19:23:39.914179 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:23:39.914153    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"119463e8fdf44c842b43b9466d71458e5c3ced5428b08d265ecd7e7272d79257\": container with ID starting with 119463e8fdf44c842b43b9466d71458e5c3ced5428b08d265ecd7e7272d79257 not found: ID does not exist" containerID="119463e8fdf44c842b43b9466d71458e5c3ced5428b08d265ecd7e7272d79257"
Oct 30 19:23:39.914290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.914187    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"119463e8fdf44c842b43b9466d71458e5c3ced5428b08d265ecd7e7272d79257"} err="failed to get container status \"119463e8fdf44c842b43b9466d71458e5c3ced5428b08d265ecd7e7272d79257\": rpc error: code = NotFound desc = could not find container \"119463e8fdf44c842b43b9466d71458e5c3ced5428b08d265ecd7e7272d79257\": container with ID starting with 119463e8fdf44c842b43b9466d71458e5c3ced5428b08d265ecd7e7272d79257 not found: ID does not exist"
Oct 30 19:23:39.914290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.914211    5704 scope.go:122] "RemoveContainer" containerID="144c673dd6ae48c606d5493dea96f3fe4c2d64d83e8b721e4cca50a54d6515ea"
Oct 30 19:23:39.924395 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.924372    5704 scope.go:122] "RemoveContainer" containerID="144c673dd6ae48c606d5493dea96f3fe4c2d64d83e8b721e4cca50a54d6515ea"
Oct 30 19:23:39.924650 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:23:39.924632    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"144c673dd6ae48c606d5493dea96f3fe4c2d64d83e8b721e4cca50a54d6515ea\": container with ID starting with 144c673dd6ae48c606d5493dea96f3fe4c2d64d83e8b721e4cca50a54d6515ea not found: ID does not exist" containerID="144c673dd6ae48c606d5493dea96f3fe4c2d64d83e8b721e4cca50a54d6515ea"
Oct 30 19:23:39.924708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.924656    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"144c673dd6ae48c606d5493dea96f3fe4c2d64d83e8b721e4cca50a54d6515ea"} err="failed to get container status \"144c673dd6ae48c606d5493dea96f3fe4c2d64d83e8b721e4cca50a54d6515ea\": rpc error: code = NotFound desc = could not find container \"144c673dd6ae48c606d5493dea96f3fe4c2d64d83e8b721e4cca50a54d6515ea\": container with ID starting with 144c673dd6ae48c606d5493dea96f3fe4c2d64d83e8b721e4cca50a54d6515ea not found: ID does not exist"
Oct 30 19:23:39.980388 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:39.980359    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxzvf\" (UniqueName: \"kubernetes.io/projected/60815cd9-d702-4e39-a59b-763096934ca6-kube-api-access-vxzvf\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:23:40.058840 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:40.058793    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"]
Oct 30 19:23:40.062887 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:40.062527    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-fb4a1b-79bfc9d875-jwqjj"]
Oct 30 19:23:40.165962 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:40.165875    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"]
Oct 30 19:23:40.170240 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:40.170214    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-fb4a1b-54c878c746-q2tlj"]
Oct 30 19:23:40.720087 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:40.720047    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:40.941896 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:40.941853    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:41.125447 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:41.125402    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:41.179961 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:41.179934    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="60815cd9-d702-4e39-a59b-763096934ca6" path="/var/lib/kubelet/pods/60815cd9-d702-4e39-a59b-763096934ca6/volumes"
Oct 30 19:23:41.180400 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:41.180386    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="79d9bcef-ed50-4e58-9a38-ab11f215ada6" path="/var/lib/kubelet/pods/79d9bcef-ed50-4e58-9a38-ab11f215ada6/volumes"
Oct 30 19:23:41.786399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:41.786365    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:46.009976 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:46.009930    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:48.178947 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:48.178911    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:48.363594 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:48.363560    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:48.554674 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:48.554636    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:50.356705 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:50.356670    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:50.538504 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:50.538472    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:50.824729 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:50.824685    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:55.740095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:55.733929    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:56.712403 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:56.712365    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:23:56.942755 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:23:56.942721    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:00.612005 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:00.611969    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:01.926293 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:01.926261    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:24:03.383728 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:24:03.383697    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:24:04.289780 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:04.289745    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:04.757090 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:04.757059    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:05.491812 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:05.491775    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:05.671394 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:05.671355    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:05.861915 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:05.861882    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:06.608024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.607992    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"]
Oct 30 19:24:06.608589 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608575    5704 cpu_manager.go:414] "containerMap: removing container" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerName="istio-proxy"
Oct 30 19:24:06.608635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608590    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerName="istio-proxy"
Oct 30 19:24:06.608635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608601    5704 cpu_manager.go:414] "containerMap: removing container" podUID="79d9bcef-ed50-4e58-9a38-ab11f215ada6" containerName="maas-api"
Oct 30 19:24:06.608635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608607    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="79d9bcef-ed50-4e58-9a38-ab11f215ada6" containerName="maas-api"
Oct 30 19:24:06.608635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608619    5704 cpu_manager.go:414] "containerMap: removing container" podUID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" containerName="istio-proxy"
Oct 30 19:24:06.608635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608625    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" containerName="istio-proxy"
Oct 30 19:24:06.608786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608645    5704 cpu_manager.go:414] "containerMap: removing container" podUID="6d681bc0-72b9-4624-8978-48afb725d311" containerName="payload-pre-processing"
Oct 30 19:24:06.608786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608652    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6d681bc0-72b9-4624-8978-48afb725d311" containerName="payload-pre-processing"
Oct 30 19:24:06.608786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608664    5704 cpu_manager.go:414] "containerMap: removing container" podUID="af419107-fbad-4021-88a0-aef9c988537e" containerName="payload-processing"
Oct 30 19:24:06.608786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608669    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af419107-fbad-4021-88a0-aef9c988537e" containerName="payload-processing"
Oct 30 19:24:06.608786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608675    5704 cpu_manager.go:414] "containerMap: removing container" podUID="60815cd9-d702-4e39-a59b-763096934ca6" containerName="payload-processing"
Oct 30 19:24:06.608786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608679    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="60815cd9-d702-4e39-a59b-763096934ca6" containerName="payload-processing"
Oct 30 19:24:06.608786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608692    5704 cpu_manager.go:414] "containerMap: removing container" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerName="istio-proxy"
Oct 30 19:24:06.608786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608696    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerName="istio-proxy"
Oct 30 19:24:06.608786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608783    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="d494fbe9-42fe-4523-b7c2-a6b1771c18e3" containerName="istio-proxy"
Oct 30 19:24:06.609094 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608790    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="79d9bcef-ed50-4e58-9a38-ab11f215ada6" containerName="maas-api"
Oct 30 19:24:06.609094 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608797    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0b50b44-9222-4120-ab91-f11dfd78f7e5" containerName="istio-proxy"
Oct 30 19:24:06.609094 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608807    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="6d681bc0-72b9-4624-8978-48afb725d311" containerName="payload-pre-processing"
Oct 30 19:24:06.609094 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608815    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="af419107-fbad-4021-88a0-aef9c988537e" containerName="payload-processing"
Oct 30 19:24:06.609094 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608820    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="60815cd9-d702-4e39-a59b-763096934ca6" containerName="payload-processing"
Oct 30 19:24:06.609094 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.608825    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="3d4c56c2-a3e9-4df0-bc32-14dcab3e91d2" containerName="istio-proxy"
Oct 30 19:24:06.611560 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.611540    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"
Oct 30 19:24:06.682574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.682528    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-65d741-dockercfg-srq42\""
Oct 30 19:24:06.687481 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.687452    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:06.728837 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.728808    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-65d741\""
Oct 30 19:24:06.732754 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.732726    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"]
Oct 30 19:24:06.738748 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.738714    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1601a23-48a1-48ea-b411-12add8a9c961-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf\" (UID: \"e1601a23-48a1-48ea-b411-12add8a9c961\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"
Oct 30 19:24:06.739222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.739200    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5jrt\" (UniqueName: \"kubernetes.io/projected/e1601a23-48a1-48ea-b411-12add8a9c961-kube-api-access-j5jrt\") pod \"payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf\" (UID: \"e1601a23-48a1-48ea-b411-12add8a9c961\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"
Oct 30 19:24:06.840064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.840013    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5jrt\" (UniqueName: \"kubernetes.io/projected/e1601a23-48a1-48ea-b411-12add8a9c961-kube-api-access-j5jrt\") pod \"payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf\" (UID: \"e1601a23-48a1-48ea-b411-12add8a9c961\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"
Oct 30 19:24:06.840250 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.840132    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1601a23-48a1-48ea-b411-12add8a9c961-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf\" (UID: \"e1601a23-48a1-48ea-b411-12add8a9c961\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"
Oct 30 19:24:06.840832 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.840807    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1601a23-48a1-48ea-b411-12add8a9c961-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf\" (UID: \"e1601a23-48a1-48ea-b411-12add8a9c961\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"
Oct 30 19:24:06.859072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.858974    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:06.917106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.917023    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5jrt\" (UniqueName: \"kubernetes.io/projected/e1601a23-48a1-48ea-b411-12add8a9c961-kube-api-access-j5jrt\") pod \"payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf\" (UID: \"e1601a23-48a1-48ea-b411-12add8a9c961\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"
Oct 30 19:24:06.924329 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:06.924290    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"
Oct 30 19:24:07.291671 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:07.291622    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:07.307785 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:07.307749    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"]
Oct 30 19:24:07.313709 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:24:07.313685    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1601a23_48a1_48ea_b411_12add8a9c961.slice/crio-826cededcf23f9e2d219f17c5d71f30d08b990da43c05718b0729700cbb28dcd WatchSource:0}: Error finding container 826cededcf23f9e2d219f17c5d71f30d08b990da43c05718b0729700cbb28dcd: Status 404 returned error can't find the container with id 826cededcf23f9e2d219f17c5d71f30d08b990da43c05718b0729700cbb28dcd
Oct 30 19:24:08.103568 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:08.103490    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf" event={"ID":"e1601a23-48a1-48ea-b411-12add8a9c961","Type":"ContainerStarted","Data":"236fc5377f2512aa851d1cf5d9465634c888e314d5f54979f5618e7cc9176a93"}
Oct 30 19:24:08.103938 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:08.103565    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf" event={"ID":"e1601a23-48a1-48ea-b411-12add8a9c961","Type":"ContainerStarted","Data":"826cededcf23f9e2d219f17c5d71f30d08b990da43c05718b0729700cbb28dcd"}
Oct 30 19:24:08.103938 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:08.103605    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"
Oct 30 19:24:08.214941 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:08.214891    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf" podStartSLOduration=2.214877831 podStartE2EDuration="2.214877831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:24:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:24:08.213395594 +0000 UTC m=+2221.159563496" watchObservedRunningTime="2026-10-30 19:24:08.214877831 +0000 UTC m=+2221.161045755"
Oct 30 19:24:11.862829 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:11.862793    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:16.433780 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:16.423294    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:19.112186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:19.112157    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"
Oct 30 19:24:21.119239 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:21.119200    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:27.338058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:27.334765    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:32.260675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:32.260638    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:33.383316 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:24:33.383279    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:24:33.875797 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:33.875764    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:34.847622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:34.847587    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:35.799434 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:35.799403    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:36.030078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:36.030019    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:36.273548 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:36.273514    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:38.891793 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:38.891755    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:43.922168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:43.922136    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:48.699438 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:48.699287    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:50.034684 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:50.034643    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"]
Oct 30 19:24:50.442339 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:50.442222    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:50.672500 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:50.672449    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:52.276695 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:52.276660    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:54.905113 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:54.905078    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:24:55.035771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.035730    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb" podUID="ae1156ad-72a2-460f-95f5-bb4c7048ed94" containerName="payload-processing" containerID="cri-o://1152c73a0aa03b7eb8aa0e4f9d4b2e04d2197d17f91d87053831e80dbb1f7000" gracePeriod=25
Oct 30 19:24:55.306262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.306236    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"
Oct 30 19:24:55.382954 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.382923    5704 generic.go:356] "Generic (PLEG): container finished" podID="ae1156ad-72a2-460f-95f5-bb4c7048ed94" containerID="1152c73a0aa03b7eb8aa0e4f9d4b2e04d2197d17f91d87053831e80dbb1f7000" exitCode=0
Oct 30 19:24:55.383172 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.382975    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"
Oct 30 19:24:55.383172 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.383002    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb" event={"ID":"ae1156ad-72a2-460f-95f5-bb4c7048ed94","Type":"ContainerDied","Data":"1152c73a0aa03b7eb8aa0e4f9d4b2e04d2197d17f91d87053831e80dbb1f7000"}
Oct 30 19:24:55.383172 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.383073    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb" event={"ID":"ae1156ad-72a2-460f-95f5-bb4c7048ed94","Type":"ContainerDied","Data":"b33002a5ddb91f68e578c1537f75bb695ece4b94b9cfd12c61e0af5d16ae3c28"}
Oct 30 19:24:55.383172 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.383098    5704 scope.go:122] "RemoveContainer" containerID="1152c73a0aa03b7eb8aa0e4f9d4b2e04d2197d17f91d87053831e80dbb1f7000"
Oct 30 19:24:55.395395 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.395375    5704 scope.go:122] "RemoveContainer" containerID="1152c73a0aa03b7eb8aa0e4f9d4b2e04d2197d17f91d87053831e80dbb1f7000"
Oct 30 19:24:55.395680 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:24:55.395659    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1152c73a0aa03b7eb8aa0e4f9d4b2e04d2197d17f91d87053831e80dbb1f7000\": container with ID starting with 1152c73a0aa03b7eb8aa0e4f9d4b2e04d2197d17f91d87053831e80dbb1f7000 not found: ID does not exist" containerID="1152c73a0aa03b7eb8aa0e4f9d4b2e04d2197d17f91d87053831e80dbb1f7000"
Oct 30 19:24:55.395746 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.395692    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1152c73a0aa03b7eb8aa0e4f9d4b2e04d2197d17f91d87053831e80dbb1f7000"} err="failed to get container status \"1152c73a0aa03b7eb8aa0e4f9d4b2e04d2197d17f91d87053831e80dbb1f7000\": rpc error: code = NotFound desc = could not find container \"1152c73a0aa03b7eb8aa0e4f9d4b2e04d2197d17f91d87053831e80dbb1f7000\": container with ID starting with 1152c73a0aa03b7eb8aa0e4f9d4b2e04d2197d17f91d87053831e80dbb1f7000 not found: ID does not exist"
Oct 30 19:24:55.410179 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.410159    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ae1156ad-72a2-460f-95f5-bb4c7048ed94-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae1156ad-72a2-460f-95f5-bb4c7048ed94-plugins-config-volume\") pod \"ae1156ad-72a2-460f-95f5-bb4c7048ed94\" (UID: \"ae1156ad-72a2-460f-95f5-bb4c7048ed94\") "
Oct 30 19:24:55.410284 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.410272    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ae1156ad-72a2-460f-95f5-bb4c7048ed94-kube-api-access-dq5hv\" (UniqueName: \"kubernetes.io/projected/ae1156ad-72a2-460f-95f5-bb4c7048ed94-kube-api-access-dq5hv\") pod \"ae1156ad-72a2-460f-95f5-bb4c7048ed94\" (UID: \"ae1156ad-72a2-460f-95f5-bb4c7048ed94\") "
Oct 30 19:24:55.410499 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.410480    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae1156ad-72a2-460f-95f5-bb4c7048ed94-plugins-config-volume" pod "ae1156ad-72a2-460f-95f5-bb4c7048ed94" (UID: "ae1156ad-72a2-460f-95f5-bb4c7048ed94"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:24:55.410721 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.410707    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ae1156ad-72a2-460f-95f5-bb4c7048ed94-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:55.412258 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.412236    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae1156ad-72a2-460f-95f5-bb4c7048ed94-kube-api-access-dq5hv" pod "ae1156ad-72a2-460f-95f5-bb4c7048ed94" (UID: "ae1156ad-72a2-460f-95f5-bb4c7048ed94"). InnerVolumeSpecName "kube-api-access-dq5hv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:24:55.512017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.511986    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dq5hv\" (UniqueName: \"kubernetes.io/projected/ae1156ad-72a2-460f-95f5-bb4c7048ed94-kube-api-access-dq5hv\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:24:55.798152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.798071    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"]
Oct 30 19:24:55.843964 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:55.843931    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-324ebe-587bf6794d-fnqxb"]
Oct 30 19:24:57.180371 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:57.180338    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ae1156ad-72a2-460f-95f5-bb4c7048ed94" path="/var/lib/kubelet/pods/ae1156ad-72a2-460f-95f5-bb4c7048ed94/volumes"
Oct 30 19:24:59.742656 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:24:59.742612    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:02.926569 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:02.926529    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:03.160150 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:03.160112    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:03.352487 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:03.342229    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:03.406213 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:25:03.406177    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:25:03.621518 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:03.621443    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:04.235635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.235605    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"]
Oct 30 19:25:04.236226 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.236212    5704 cpu_manager.go:414] "containerMap: removing container" podUID="ae1156ad-72a2-460f-95f5-bb4c7048ed94" containerName="payload-processing"
Oct 30 19:25:04.236226 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.236226    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ae1156ad-72a2-460f-95f5-bb4c7048ed94" containerName="payload-processing"
Oct 30 19:25:04.236318 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.236308    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="ae1156ad-72a2-460f-95f5-bb4c7048ed94" containerName="payload-processing"
Oct 30 19:25:04.239570 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.239551    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"
Oct 30 19:25:04.334131 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.334096    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-539084-dockercfg-kb9h8\""
Oct 30 19:25:04.334321 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.334100    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-539084\""
Oct 30 19:25:04.336508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.336482    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"]
Oct 30 19:25:04.371484 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.371461    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1d49546-da6c-445c-a491-3205a4daf0a8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r\" (UID: \"f1d49546-da6c-445c-a491-3205a4daf0a8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"
Oct 30 19:25:04.371595 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.371530    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9kkb\" (UniqueName: \"kubernetes.io/projected/f1d49546-da6c-445c-a491-3205a4daf0a8-kube-api-access-v9kkb\") pod \"payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r\" (UID: \"f1d49546-da6c-445c-a491-3205a4daf0a8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"
Oct 30 19:25:04.472498 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.472468    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1d49546-da6c-445c-a491-3205a4daf0a8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r\" (UID: \"f1d49546-da6c-445c-a491-3205a4daf0a8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"
Oct 30 19:25:04.472657 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.472548    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9kkb\" (UniqueName: \"kubernetes.io/projected/f1d49546-da6c-445c-a491-3205a4daf0a8-kube-api-access-v9kkb\") pod \"payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r\" (UID: \"f1d49546-da6c-445c-a491-3205a4daf0a8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"
Oct 30 19:25:04.473169 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.473150    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1d49546-da6c-445c-a491-3205a4daf0a8-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r\" (UID: \"f1d49546-da6c-445c-a491-3205a4daf0a8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"
Oct 30 19:25:04.530170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.530146    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9kkb\" (UniqueName: \"kubernetes.io/projected/f1d49546-da6c-445c-a491-3205a4daf0a8-kube-api-access-v9kkb\") pod \"payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r\" (UID: \"f1d49546-da6c-445c-a491-3205a4daf0a8\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"
Oct 30 19:25:04.552689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.552666    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"
Oct 30 19:25:04.930998 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:04.930916    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"]
Oct 30 19:25:05.452577 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:05.452544    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r" event={"ID":"f1d49546-da6c-445c-a491-3205a4daf0a8","Type":"ContainerStarted","Data":"b9b66ff89396e73438b42c0c8e03194aaea6595764490af59184caee50330b2b"}
Oct 30 19:25:05.453078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:05.452594    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r" event={"ID":"f1d49546-da6c-445c-a491-3205a4daf0a8","Type":"ContainerStarted","Data":"72e055d0643478f1abd262dfd5ccd26dd800f58dcc23f81cd60855faa78fc4ae"}
Oct 30 19:25:05.453078 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:05.452631    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"
Oct 30 19:25:05.564068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:05.564003    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r" podStartSLOduration=1.563989252 podStartE2EDuration="1.563989252s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:05.5625808 +0000 UTC m=+2278.508748702" watchObservedRunningTime="2026-10-30 19:25:05.563989252 +0000 UTC m=+2278.510157206"
Oct 30 19:25:06.243423 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:06.243380    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:07.164382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:07.164353    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:07.826836 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:07.826806    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:07.836918 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:07.836892    5704 scope.go:122] "RemoveContainer" containerID="4fa705dc7a2071416f602dce5501325d9be8cbf9337a0b23d0adee2d92c67114"
Oct 30 19:25:08.062802 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:08.062775    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:10.962503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:10.962466    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:11.206321 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.206290    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:11.291338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.291307    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"]
Oct 30 19:25:11.296866 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.296847    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"
Oct 30 19:25:11.345560 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.345539    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-2a5c13-dockercfg-m9q27\""
Oct 30 19:25:11.414058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.394449    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-2a5c13\""
Oct 30 19:25:11.432849 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.432798    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"]
Oct 30 19:25:11.479469 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.478208    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:11.481266 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.480545    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qz6qk\" (UniqueName: \"kubernetes.io/projected/a37355ab-b5c4-48af-b492-6536e09680dc-kube-api-access-qz6qk\") pod \"payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm\" (UID: \"a37355ab-b5c4-48af-b492-6536e09680dc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"
Oct 30 19:25:11.481266 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.480618    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a37355ab-b5c4-48af-b492-6536e09680dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm\" (UID: \"a37355ab-b5c4-48af-b492-6536e09680dc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"
Oct 30 19:25:11.582217 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.582138    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qz6qk\" (UniqueName: \"kubernetes.io/projected/a37355ab-b5c4-48af-b492-6536e09680dc-kube-api-access-qz6qk\") pod \"payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm\" (UID: \"a37355ab-b5c4-48af-b492-6536e09680dc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"
Oct 30 19:25:11.582217 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.582201    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a37355ab-b5c4-48af-b492-6536e09680dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm\" (UID: \"a37355ab-b5c4-48af-b492-6536e09680dc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"
Oct 30 19:25:11.582864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.582846    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a37355ab-b5c4-48af-b492-6536e09680dc-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm\" (UID: \"a37355ab-b5c4-48af-b492-6536e09680dc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"
Oct 30 19:25:11.640251 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.640216    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qz6qk\" (UniqueName: \"kubernetes.io/projected/a37355ab-b5c4-48af-b492-6536e09680dc-kube-api-access-qz6qk\") pod \"payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm\" (UID: \"a37355ab-b5c4-48af-b492-6536e09680dc\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"
Oct 30 19:25:11.910391 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:11.910304    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"
Oct 30 19:25:12.286713 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:12.286692    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"]
Oct 30 19:25:12.289292 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:25:12.289262    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda37355ab_b5c4_48af_b492_6536e09680dc.slice/crio-ba7d2e35f55d960dd3d1599b9c3eb016a90a5047b1ebb80c69ccfa7b1daa5874 WatchSource:0}: Error finding container ba7d2e35f55d960dd3d1599b9c3eb016a90a5047b1ebb80c69ccfa7b1daa5874: Status 404 returned error can't find the container with id ba7d2e35f55d960dd3d1599b9c3eb016a90a5047b1ebb80c69ccfa7b1daa5874
Oct 30 19:25:12.505627 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:12.505588    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm" event={"ID":"a37355ab-b5c4-48af-b492-6536e09680dc","Type":"ContainerStarted","Data":"b70a803f0b1a054af53f9612bcce36ca3377d9eeacf33a6a8981746a9613c65d"}
Oct 30 19:25:12.505816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:12.505651    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm" event={"ID":"a37355ab-b5c4-48af-b492-6536e09680dc","Type":"ContainerStarted","Data":"ba7d2e35f55d960dd3d1599b9c3eb016a90a5047b1ebb80c69ccfa7b1daa5874"}
Oct 30 19:25:12.505816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:12.505719    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"
Oct 30 19:25:12.621914 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:12.621818    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm" podStartSLOduration=1.6218041250000002 podStartE2EDuration="1.621804125s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:12.619345589 +0000 UTC m=+2285.565513492" watchObservedRunningTime="2026-10-30 19:25:12.621804125 +0000 UTC m=+2285.567972050"
Oct 30 19:25:14.739983 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:14.739944    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:15.248520 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:15.248484    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:15.833477 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:15.833447    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"]
Oct 30 19:25:15.837735 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:15.837716    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"
Oct 30 19:25:15.886543 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:15.886525    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-496c5b\""
Oct 30 19:25:15.932794 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:15.932767    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-496c5b-dockercfg-s9wss\""
Oct 30 19:25:15.935576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:15.935553    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"]
Oct 30 19:25:15.950318 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:15.950293    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9d79b71-6123-441f-9464-41e50708ce97-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb\" (UID: \"d9d79b71-6123-441f-9464-41e50708ce97\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"
Oct 30 19:25:15.950412 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:15.950351    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7m2g\" (UniqueName: \"kubernetes.io/projected/d9d79b71-6123-441f-9464-41e50708ce97-kube-api-access-r7m2g\") pod \"payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb\" (UID: \"d9d79b71-6123-441f-9464-41e50708ce97\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"
Oct 30 19:25:16.052003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:16.051979    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9d79b71-6123-441f-9464-41e50708ce97-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb\" (UID: \"d9d79b71-6123-441f-9464-41e50708ce97\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"
Oct 30 19:25:16.052148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:16.052016    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7m2g\" (UniqueName: \"kubernetes.io/projected/d9d79b71-6123-441f-9464-41e50708ce97-kube-api-access-r7m2g\") pod \"payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb\" (UID: \"d9d79b71-6123-441f-9464-41e50708ce97\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"
Oct 30 19:25:16.052653 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:16.052633    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9d79b71-6123-441f-9464-41e50708ce97-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb\" (UID: \"d9d79b71-6123-441f-9464-41e50708ce97\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"
Oct 30 19:25:16.110321 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:16.110258    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7m2g\" (UniqueName: \"kubernetes.io/projected/d9d79b71-6123-441f-9464-41e50708ce97-kube-api-access-r7m2g\") pod \"payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb\" (UID: \"d9d79b71-6123-441f-9464-41e50708ce97\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"
Oct 30 19:25:16.151520 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:16.151495    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"
Oct 30 19:25:16.460979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:16.460900    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"
Oct 30 19:25:16.521198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:16.521178    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"]
Oct 30 19:25:16.523658 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:25:16.523628    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9d79b71_6123_441f_9464_41e50708ce97.slice/crio-bea51d41fc1771083c6f7559aef9a0f27d42d08763cb1d26936af6a4e445b0ff WatchSource:0}: Error finding container bea51d41fc1771083c6f7559aef9a0f27d42d08763cb1d26936af6a4e445b0ff: Status 404 returned error can't find the container with id bea51d41fc1771083c6f7559aef9a0f27d42d08763cb1d26936af6a4e445b0ff
Oct 30 19:25:17.128065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.127990    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:17.430327 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.430247    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"]
Oct 30 19:25:17.430498 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.430445    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf" podUID="e1601a23-48a1-48ea-b411-12add8a9c961" containerName="payload-processing" containerID="cri-o://236fc5377f2512aa851d1cf5d9465634c888e314d5f54979f5618e7cc9176a93" gracePeriod=30
Oct 30 19:25:17.579425 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.579375    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb" event={"ID":"d9d79b71-6123-441f-9464-41e50708ce97","Type":"ContainerStarted","Data":"9627685dfa1f6298f0945588583a52f53414044335e4285a75b687e52a567a83"}
Oct 30 19:25:17.579612 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.579476    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb" event={"ID":"d9d79b71-6123-441f-9464-41e50708ce97","Type":"ContainerStarted","Data":"bea51d41fc1771083c6f7559aef9a0f27d42d08763cb1d26936af6a4e445b0ff"}
Oct 30 19:25:17.579612 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.579508    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"
Oct 30 19:25:17.624972 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.624940    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:17.693027 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.692932    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb" podStartSLOduration=2.692912219 podStartE2EDuration="2.692912219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:17.689012131 +0000 UTC m=+2290.635180037" watchObservedRunningTime="2026-10-30 19:25:17.692912219 +0000 UTC m=+2290.639080122"
Oct 30 19:25:17.733609 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.733588    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"
Oct 30 19:25:17.862326 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.862295    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:17.873321 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.873298    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1601a23-48a1-48ea-b411-12add8a9c961-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1601a23-48a1-48ea-b411-12add8a9c961-plugins-config-volume\") pod \"e1601a23-48a1-48ea-b411-12add8a9c961\" (UID: \"e1601a23-48a1-48ea-b411-12add8a9c961\") "
Oct 30 19:25:17.873520 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.873507    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1601a23-48a1-48ea-b411-12add8a9c961-kube-api-access-j5jrt\" (UniqueName: \"kubernetes.io/projected/e1601a23-48a1-48ea-b411-12add8a9c961-kube-api-access-j5jrt\") pod \"e1601a23-48a1-48ea-b411-12add8a9c961\" (UID: \"e1601a23-48a1-48ea-b411-12add8a9c961\") "
Oct 30 19:25:17.873775 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.873747    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1601a23-48a1-48ea-b411-12add8a9c961-plugins-config-volume" pod "e1601a23-48a1-48ea-b411-12add8a9c961" (UID: "e1601a23-48a1-48ea-b411-12add8a9c961"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:25:17.873869 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.873842    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e1601a23-48a1-48ea-b411-12add8a9c961-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:17.875591 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.875565    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1601a23-48a1-48ea-b411-12add8a9c961-kube-api-access-j5jrt" pod "e1601a23-48a1-48ea-b411-12add8a9c961" (UID: "e1601a23-48a1-48ea-b411-12add8a9c961"). InnerVolumeSpecName "kube-api-access-j5jrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:25:17.975145 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:17.975076    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j5jrt\" (UniqueName: \"kubernetes.io/projected/e1601a23-48a1-48ea-b411-12add8a9c961-kube-api-access-j5jrt\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:25:18.586529 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:18.586490    5704 generic.go:356] "Generic (PLEG): container finished" podID="e1601a23-48a1-48ea-b411-12add8a9c961" containerID="236fc5377f2512aa851d1cf5d9465634c888e314d5f54979f5618e7cc9176a93" exitCode=0
Oct 30 19:25:18.586987 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:18.586540    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"
Oct 30 19:25:18.586987 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:18.586581    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf" event={"ID":"e1601a23-48a1-48ea-b411-12add8a9c961","Type":"ContainerDied","Data":"236fc5377f2512aa851d1cf5d9465634c888e314d5f54979f5618e7cc9176a93"}
Oct 30 19:25:18.586987 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:18.586653    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf" event={"ID":"e1601a23-48a1-48ea-b411-12add8a9c961","Type":"ContainerDied","Data":"826cededcf23f9e2d219f17c5d71f30d08b990da43c05718b0729700cbb28dcd"}
Oct 30 19:25:18.586987 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:18.586690    5704 scope.go:122] "RemoveContainer" containerID="236fc5377f2512aa851d1cf5d9465634c888e314d5f54979f5618e7cc9176a93"
Oct 30 19:25:18.597142 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:18.597120    5704 scope.go:122] "RemoveContainer" containerID="236fc5377f2512aa851d1cf5d9465634c888e314d5f54979f5618e7cc9176a93"
Oct 30 19:25:18.597375 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:25:18.597356    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"236fc5377f2512aa851d1cf5d9465634c888e314d5f54979f5618e7cc9176a93\": container with ID starting with 236fc5377f2512aa851d1cf5d9465634c888e314d5f54979f5618e7cc9176a93 not found: ID does not exist" containerID="236fc5377f2512aa851d1cf5d9465634c888e314d5f54979f5618e7cc9176a93"
Oct 30 19:25:18.597421 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:18.597383    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"236fc5377f2512aa851d1cf5d9465634c888e314d5f54979f5618e7cc9176a93"} err="failed to get container status \"236fc5377f2512aa851d1cf5d9465634c888e314d5f54979f5618e7cc9176a93\": rpc error: code = NotFound desc = could not find container \"236fc5377f2512aa851d1cf5d9465634c888e314d5f54979f5618e7cc9176a93\": container with ID starting with 236fc5377f2512aa851d1cf5d9465634c888e314d5f54979f5618e7cc9176a93 not found: ID does not exist"
Oct 30 19:25:18.634401 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:18.634372    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:18.741116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:18.741093    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"]
Oct 30 19:25:18.787546 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:18.787518    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-65d741-7cffd99666-dbnsf"]
Oct 30 19:25:18.884145 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:18.884064    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:19.065732 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:19.065693    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:19.179733 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:19.179660    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1601a23-48a1-48ea-b411-12add8a9c961" path="/var/lib/kubelet/pods/e1601a23-48a1-48ea-b411-12add8a9c961/volumes"
Oct 30 19:25:19.621503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:19.621471    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:23.515280 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:23.515250    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"
Oct 30 19:25:24.073886 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:24.073852    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:25.918284 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:25.918245    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:26.113009 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:26.112972    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:26.302886 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:26.302851    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:27.833291 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:27.833258    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:28.455975 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:28.455936    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:28.589310 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:28.589281    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"
Oct 30 19:25:28.695045 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:28.695005    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:29.284198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:29.284151    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:33.042132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:33.042097    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:25:33.384665 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:25:33.384585    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:25:35.585746 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:35.585706    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:40.409006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:40.408634    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:43.236214 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:43.236168    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:43.479690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:43.479659    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:46.315875 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.315839    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:46.497254 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.497220    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:46.696437 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.696359    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"]
Oct 30 19:25:46.696985 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.696970    5704 cpu_manager.go:414] "containerMap: removing container" podUID="e1601a23-48a1-48ea-b411-12add8a9c961" containerName="payload-processing"
Oct 30 19:25:46.697043 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.696985    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1601a23-48a1-48ea-b411-12add8a9c961" containerName="payload-processing"
Oct 30 19:25:46.697104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.697094    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1601a23-48a1-48ea-b411-12add8a9c961" containerName="payload-processing"
Oct 30 19:25:46.700938 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.700919    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"
Oct 30 19:25:46.773062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.770351    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-4833f3\""
Oct 30 19:25:46.817019 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.816986    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-4833f3-dockercfg-sqpn9\""
Oct 30 19:25:46.823120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.823085    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"]
Oct 30 19:25:46.883630 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.883597    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs9l4\" (UniqueName: \"kubernetes.io/projected/702caeab-5ddb-4a78-b1a2-80993766a989-kube-api-access-zs9l4\") pod \"payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l\" (UID: \"702caeab-5ddb-4a78-b1a2-80993766a989\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"
Oct 30 19:25:46.883797 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.883703    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/702caeab-5ddb-4a78-b1a2-80993766a989-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l\" (UID: \"702caeab-5ddb-4a78-b1a2-80993766a989\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"
Oct 30 19:25:46.924359 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.924177    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:46.985215 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.985138    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/702caeab-5ddb-4a78-b1a2-80993766a989-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l\" (UID: \"702caeab-5ddb-4a78-b1a2-80993766a989\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"
Oct 30 19:25:46.985359 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.985285    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs9l4\" (UniqueName: \"kubernetes.io/projected/702caeab-5ddb-4a78-b1a2-80993766a989-kube-api-access-zs9l4\") pod \"payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l\" (UID: \"702caeab-5ddb-4a78-b1a2-80993766a989\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"
Oct 30 19:25:46.985822 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:46.985802    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/702caeab-5ddb-4a78-b1a2-80993766a989-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l\" (UID: \"702caeab-5ddb-4a78-b1a2-80993766a989\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"
Oct 30 19:25:47.043209 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:47.043184    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs9l4\" (UniqueName: \"kubernetes.io/projected/702caeab-5ddb-4a78-b1a2-80993766a989-kube-api-access-zs9l4\") pod \"payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l\" (UID: \"702caeab-5ddb-4a78-b1a2-80993766a989\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"
Oct 30 19:25:47.314563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:47.314533    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"
Oct 30 19:25:47.689141 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:47.689114    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"]
Oct 30 19:25:47.691694 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:25:47.691666    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod702caeab_5ddb_4a78_b1a2_80993766a989.slice/crio-a90c99ea5a3ca43ac97fd135b6f3f23e0fb7b51b887de2d70e24cef2d5c3a44b WatchSource:0}: Error finding container a90c99ea5a3ca43ac97fd135b6f3f23e0fb7b51b887de2d70e24cef2d5c3a44b: Status 404 returned error can't find the container with id a90c99ea5a3ca43ac97fd135b6f3f23e0fb7b51b887de2d70e24cef2d5c3a44b
Oct 30 19:25:47.837005 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:47.836973    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l" event={"ID":"702caeab-5ddb-4a78-b1a2-80993766a989","Type":"ContainerStarted","Data":"5128ef619b166cd20ac37ea0a421ad542672157461cfaa13032d2a46063d1570"}
Oct 30 19:25:47.837160 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:47.837028    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l" event={"ID":"702caeab-5ddb-4a78-b1a2-80993766a989","Type":"ContainerStarted","Data":"a90c99ea5a3ca43ac97fd135b6f3f23e0fb7b51b887de2d70e24cef2d5c3a44b"}
Oct 30 19:25:47.837208 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:47.837190    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"
Oct 30 19:25:47.948409 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:47.948332    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l" podStartSLOduration=1.948320023 podStartE2EDuration="1.948320023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:25:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:25:47.946956834 +0000 UTC m=+2320.893124736" watchObservedRunningTime="2026-10-30 19:25:47.948320023 +0000 UTC m=+2320.894487926"
Oct 30 19:25:51.300019 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:51.299977    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:51.536671 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:51.536638    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:51.763213 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:51.763179    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:53.145588 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:53.145551    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:56.368675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:56.368618    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:25:58.845789 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:25:58.845759    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"
Oct 30 19:26:03.066249 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:03.066208    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:03.383690 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:26:03.383614    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:26:07.493649 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:07.493616    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:08.831283 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:08.831248    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:13.903491 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:13.903456    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"]
Oct 30 19:26:13.908772 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:13.908752    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:13.969523 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:13.969496    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-2a5c13-serving-cert\""
Oct 30 19:26:14.007051 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.000281    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhhm6\" (UniqueName: \"kubernetes.io/projected/51d91680-b377-4411-83ed-46b4ebb90ea9-kube-api-access-qhhm6\") pod \"maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:14.007051 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.000631    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-metrics-certs\") pod \"maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:14.007051 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.000856    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51d91680-b377-4411-83ed-46b4ebb90ea9-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:14.007051 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.000932    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:14.010607 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.010570    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"]
Oct 30 19:26:14.104096 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.104060    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-metrics-certs\") pod \"maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:14.104263 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.104130    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51d91680-b377-4411-83ed-46b4ebb90ea9-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:14.104263 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.104187    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:14.104263 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.104239    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhhm6\" (UniqueName: \"kubernetes.io/projected/51d91680-b377-4411-83ed-46b4ebb90ea9-kube-api-access-qhhm6\") pod \"maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:14.104412 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:26:14.104372    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-serving-cert: secret "maas-api-e2e-ipp-b-2a5c13-serving-cert" not found
Oct 30 19:26:14.104474 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:26:14.104460    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-maas-api-tls podName:51d91680-b377-4411-83ed-46b4ebb90ea9 nodeName:}" failed. No retries permitted until 2026-10-30 19:26:14.604435838 +0000 UTC m=+2347.550603725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-maas-api-tls") pod "maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84" (UID: "51d91680-b377-4411-83ed-46b4ebb90ea9") : secret "maas-api-e2e-ipp-b-2a5c13-serving-cert" not found
Oct 30 19:26:14.104925 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.104901    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51d91680-b377-4411-83ed-46b4ebb90ea9-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:14.107129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.107106    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-metrics-certs\") pod \"maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:14.162232 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.162167    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhhm6\" (UniqueName: \"kubernetes.io/projected/51d91680-b377-4411-83ed-46b4ebb90ea9-kube-api-access-qhhm6\") pod \"maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:14.188266 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.188239    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:14.413114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.413012    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:14.612956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.612923    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:14.615261 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.615238    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-maas-api-tls\") pod \"maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:14.822703 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:14.822673    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:15.207175 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:15.207151    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"]
Oct 30 19:26:15.209492 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:26:15.209463    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d91680_b377_4411_83ed_46b4ebb90ea9.slice/crio-8608dda3b98834154ad910dbfb4a6955ece40fd1e6759fb7d87034f7398776d3 WatchSource:0}: Error finding container 8608dda3b98834154ad910dbfb4a6955ece40fd1e6759fb7d87034f7398776d3: Status 404 returned error can't find the container with id 8608dda3b98834154ad910dbfb4a6955ece40fd1e6759fb7d87034f7398776d3
Oct 30 19:26:15.211116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:15.211098    5704 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:26:16.026219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:16.026186    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84" event={"ID":"51d91680-b377-4411-83ed-46b4ebb90ea9","Type":"ContainerStarted","Data":"8608dda3b98834154ad910dbfb4a6955ece40fd1e6759fb7d87034f7398776d3"}
Oct 30 19:26:17.033958 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:17.033919    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84" event={"ID":"51d91680-b377-4411-83ed-46b4ebb90ea9","Type":"ContainerStarted","Data":"db30a8112b76d69dbc12785b8e1a35e06e9598310fb9409defe4d5f0c4f40684"}
Oct 30 19:26:17.034386 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:17.034234    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:17.146190 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:17.146142    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84" podStartSLOduration=3.18082679 podStartE2EDuration="4.146127235s" totalImagesPullingTime="965.300445ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:17.144681056 +0000 UTC m=+2350.090848951" watchObservedRunningTime="2026-10-30 19:26:17.146127235 +0000 UTC m=+2350.092295138"
Oct 30 19:26:18.985943 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:18.985674    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:23.046342 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:23.046311    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:26:23.797742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:23.794643    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:25.258892 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:25.258855    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:29.981521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:29.981483    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:31.104263 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:31.104225    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:26:33.383002 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:26:33.382966    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:26:34.576157 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:34.576123    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:36.806360 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:36.806322    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:38.453804 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.453770    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"]
Oct 30 19:26:38.454269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.453979    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676" podUID="95061afb-d0ef-47d6-975e-fc5d7d500e16" containerName="maas-api" containerID="cri-o://5a4e348f490a9876e7c16abd88c0cfdf49a0673c05aaf048a7984ceb77cb481c" gracePeriod=30
Oct 30 19:26:38.599861 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.599806    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:38.759182 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.759158    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:26:38.889069 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.888977    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/95061afb-d0ef-47d6-975e-fc5d7d500e16-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95061afb-d0ef-47d6-975e-fc5d7d500e16-openshift-service-ca\") pod \"95061afb-d0ef-47d6-975e-fc5d7d500e16\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") "
Oct 30 19:26:38.889262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.889109    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-metrics-certs\" (UniqueName: \"kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-metrics-certs\") pod \"95061afb-d0ef-47d6-975e-fc5d7d500e16\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") "
Oct 30 19:26:38.889262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.889163    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-maas-api-tls\") pod \"95061afb-d0ef-47d6-975e-fc5d7d500e16\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") "
Oct 30 19:26:38.889473 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.889345    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/95061afb-d0ef-47d6-975e-fc5d7d500e16-kube-api-access-rtdkf\" (UniqueName: \"kubernetes.io/projected/95061afb-d0ef-47d6-975e-fc5d7d500e16-kube-api-access-rtdkf\") pod \"95061afb-d0ef-47d6-975e-fc5d7d500e16\" (UID: \"95061afb-d0ef-47d6-975e-fc5d7d500e16\") "
Oct 30 19:26:38.889473 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.889347    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95061afb-d0ef-47d6-975e-fc5d7d500e16-openshift-service-ca" pod "95061afb-d0ef-47d6-975e-fc5d7d500e16" (UID: "95061afb-d0ef-47d6-975e-fc5d7d500e16"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:26:38.889848 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.889820    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95061afb-d0ef-47d6-975e-fc5d7d500e16-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:38.891417 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.891394    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-maas-api-tls" pod "95061afb-d0ef-47d6-975e-fc5d7d500e16" (UID: "95061afb-d0ef-47d6-975e-fc5d7d500e16"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:26:38.891523 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.891512    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-metrics-certs" pod "95061afb-d0ef-47d6-975e-fc5d7d500e16" (UID: "95061afb-d0ef-47d6-975e-fc5d7d500e16"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:26:38.891929 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.891900    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95061afb-d0ef-47d6-975e-fc5d7d500e16-kube-api-access-rtdkf" pod "95061afb-d0ef-47d6-975e-fc5d7d500e16" (UID: "95061afb-d0ef-47d6-975e-fc5d7d500e16"). InnerVolumeSpecName "kube-api-access-rtdkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:26:38.955123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.955099    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"]
Oct 30 19:26:38.955313 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.955292    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq" podUID="c5c5d61a-a137-469c-8da5-3ceb9596ece2" containerName="payload-pre-processing" containerID="cri-o://5e8c35c540976263ee3b0648ff8c39760222fc3c33118475894a4e7fb29a8b94" gracePeriod=30
Oct 30 19:26:38.991882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.991810    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtdkf\" (UniqueName: \"kubernetes.io/projected/95061afb-d0ef-47d6-975e-fc5d7d500e16-kube-api-access-rtdkf\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:38.991882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.991848    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:38.991882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:38.991865    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95061afb-d0ef-47d6-975e-fc5d7d500e16-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:39.078883 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.078851    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:39.177465 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.177428    5704 generic.go:356] "Generic (PLEG): container finished" podID="c5c5d61a-a137-469c-8da5-3ceb9596ece2" containerID="5e8c35c540976263ee3b0648ff8c39760222fc3c33118475894a4e7fb29a8b94" exitCode=0
Oct 30 19:26:39.179024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.178992    5704 generic.go:356] "Generic (PLEG): container finished" podID="95061afb-d0ef-47d6-975e-fc5d7d500e16" containerID="5a4e348f490a9876e7c16abd88c0cfdf49a0673c05aaf048a7984ceb77cb481c" exitCode=0
Oct 30 19:26:39.179174 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.179110    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"
Oct 30 19:26:39.181994 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.181973    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq" event={"ID":"c5c5d61a-a137-469c-8da5-3ceb9596ece2","Type":"ContainerDied","Data":"5e8c35c540976263ee3b0648ff8c39760222fc3c33118475894a4e7fb29a8b94"}
Oct 30 19:26:39.182105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.182012    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676" event={"ID":"95061afb-d0ef-47d6-975e-fc5d7d500e16","Type":"ContainerDied","Data":"5a4e348f490a9876e7c16abd88c0cfdf49a0673c05aaf048a7984ceb77cb481c"}
Oct 30 19:26:39.182105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.182061    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676" event={"ID":"95061afb-d0ef-47d6-975e-fc5d7d500e16","Type":"ContainerDied","Data":"ada40ace1345282b9fc78c6fa4a0f3018a36aed73cc52c0bbe91998bf48d825a"}
Oct 30 19:26:39.182105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.182084    5704 scope.go:122] "RemoveContainer" containerID="5a4e348f490a9876e7c16abd88c0cfdf49a0673c05aaf048a7984ceb77cb481c"
Oct 30 19:26:39.192495 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.192477    5704 scope.go:122] "RemoveContainer" containerID="5a4e348f490a9876e7c16abd88c0cfdf49a0673c05aaf048a7984ceb77cb481c"
Oct 30 19:26:39.192767 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:26:39.192745    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a4e348f490a9876e7c16abd88c0cfdf49a0673c05aaf048a7984ceb77cb481c\": container with ID starting with 5a4e348f490a9876e7c16abd88c0cfdf49a0673c05aaf048a7984ceb77cb481c not found: ID does not exist" containerID="5a4e348f490a9876e7c16abd88c0cfdf49a0673c05aaf048a7984ceb77cb481c"
Oct 30 19:26:39.192862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.192773    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a4e348f490a9876e7c16abd88c0cfdf49a0673c05aaf048a7984ceb77cb481c"} err="failed to get container status \"5a4e348f490a9876e7c16abd88c0cfdf49a0673c05aaf048a7984ceb77cb481c\": rpc error: code = NotFound desc = could not find container \"5a4e348f490a9876e7c16abd88c0cfdf49a0673c05aaf048a7984ceb77cb481c\": container with ID starting with 5a4e348f490a9876e7c16abd88c0cfdf49a0673c05aaf048a7984ceb77cb481c not found: ID does not exist"
Oct 30 19:26:39.200321 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.200304    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"
Oct 30 19:26:39.270162 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.270132    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:39.295626 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.295602    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c5c5d61a-a137-469c-8da5-3ceb9596ece2-kube-api-access-m7gkh\" (UniqueName: \"kubernetes.io/projected/c5c5d61a-a137-469c-8da5-3ceb9596ece2-kube-api-access-m7gkh\") pod \"c5c5d61a-a137-469c-8da5-3ceb9596ece2\" (UID: \"c5c5d61a-a137-469c-8da5-3ceb9596ece2\") "
Oct 30 19:26:39.295749 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.295737    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c5c5d61a-a137-469c-8da5-3ceb9596ece2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5c5d61a-a137-469c-8da5-3ceb9596ece2-plugins-config-volume\") pod \"c5c5d61a-a137-469c-8da5-3ceb9596ece2\" (UID: \"c5c5d61a-a137-469c-8da5-3ceb9596ece2\") "
Oct 30 19:26:39.296116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.296093    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5c5d61a-a137-469c-8da5-3ceb9596ece2-plugins-config-volume" pod "c5c5d61a-a137-469c-8da5-3ceb9596ece2" (UID: "c5c5d61a-a137-469c-8da5-3ceb9596ece2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:26:39.296229 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.296212    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c5c5d61a-a137-469c-8da5-3ceb9596ece2-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:39.297571 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.297553    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5c5d61a-a137-469c-8da5-3ceb9596ece2-kube-api-access-m7gkh" pod "c5c5d61a-a137-469c-8da5-3ceb9596ece2" (UID: "c5c5d61a-a137-469c-8da5-3ceb9596ece2"). InnerVolumeSpecName "kube-api-access-m7gkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:26:39.412597 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.412560    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7gkh\" (UniqueName: \"kubernetes.io/projected/c5c5d61a-a137-469c-8da5-3ceb9596ece2-kube-api-access-m7gkh\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:39.460415 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.453826    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"]
Oct 30 19:26:39.460415 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.459897    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-905566-58c8b66f64-bs676"]
Oct 30 19:26:39.615741 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.615657    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"]
Oct 30 19:26:39.615903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.615860    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh" podUID="7cae896c-360e-492f-b987-dec361cf72e4" containerName="maas-api" containerID="cri-o://2bd5c17246c59f16d9135de9e71196240dd7d5383bc40ff204bc6dbd52780d71" gracePeriod=30
Oct 30 19:26:39.765811 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.765778    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:39.900478 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:39.900452    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:26:40.030056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.021650    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-maas-api-tls\") pod \"7cae896c-360e-492f-b987-dec361cf72e4\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") "
Oct 30 19:26:40.030056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.021789    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7cae896c-360e-492f-b987-dec361cf72e4-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cae896c-360e-492f-b987-dec361cf72e4-openshift-service-ca\") pod \"7cae896c-360e-492f-b987-dec361cf72e4\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") "
Oct 30 19:26:40.030056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.021861    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-metrics-certs\") pod \"7cae896c-360e-492f-b987-dec361cf72e4\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") "
Oct 30 19:26:40.030056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.021890    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7cae896c-360e-492f-b987-dec361cf72e4-kube-api-access-c4pb4\" (UniqueName: \"kubernetes.io/projected/7cae896c-360e-492f-b987-dec361cf72e4-kube-api-access-c4pb4\") pod \"7cae896c-360e-492f-b987-dec361cf72e4\" (UID: \"7cae896c-360e-492f-b987-dec361cf72e4\") "
Oct 30 19:26:40.031186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.030835    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7cae896c-360e-492f-b987-dec361cf72e4-openshift-service-ca" pod "7cae896c-360e-492f-b987-dec361cf72e4" (UID: "7cae896c-360e-492f-b987-dec361cf72e4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:26:40.040403 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.040363    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7cae896c-360e-492f-b987-dec361cf72e4-kube-api-access-c4pb4" pod "7cae896c-360e-492f-b987-dec361cf72e4" (UID: "7cae896c-360e-492f-b987-dec361cf72e4"). InnerVolumeSpecName "kube-api-access-c4pb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:26:40.042649 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.042619    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-metrics-certs" pod "7cae896c-360e-492f-b987-dec361cf72e4" (UID: "7cae896c-360e-492f-b987-dec361cf72e4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:26:40.042649 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.042633    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-maas-api-tls" pod "7cae896c-360e-492f-b987-dec361cf72e4" (UID: "7cae896c-360e-492f-b987-dec361cf72e4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:26:40.115017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.114984    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:40.131104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.128752    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:40.131104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.128785    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c4pb4\" (UniqueName: \"kubernetes.io/projected/7cae896c-360e-492f-b987-dec361cf72e4-kube-api-access-c4pb4\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:40.131104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.128801    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7cae896c-360e-492f-b987-dec361cf72e4-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:40.131104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.128817    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7cae896c-360e-492f-b987-dec361cf72e4-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:40.187300 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.187266    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"
Oct 30 19:26:40.187300 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.187281    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq" event={"ID":"c5c5d61a-a137-469c-8da5-3ceb9596ece2","Type":"ContainerDied","Data":"5588fc5d59a5676ef3eb8a7f3fcc50c20a8208c6e05631c6602ac6196251eb7b"}
Oct 30 19:26:40.187563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.187350    5704 scope.go:122] "RemoveContainer" containerID="5e8c35c540976263ee3b0648ff8c39760222fc3c33118475894a4e7fb29a8b94"
Oct 30 19:26:40.188782 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.188761    5704 generic.go:356] "Generic (PLEG): container finished" podID="7cae896c-360e-492f-b987-dec361cf72e4" containerID="2bd5c17246c59f16d9135de9e71196240dd7d5383bc40ff204bc6dbd52780d71" exitCode=0
Oct 30 19:26:40.188893 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.188798    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh" event={"ID":"7cae896c-360e-492f-b987-dec361cf72e4","Type":"ContainerDied","Data":"2bd5c17246c59f16d9135de9e71196240dd7d5383bc40ff204bc6dbd52780d71"}
Oct 30 19:26:40.188893 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.188810    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"
Oct 30 19:26:40.188893 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.188838    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh" event={"ID":"7cae896c-360e-492f-b987-dec361cf72e4","Type":"ContainerDied","Data":"e0b914fe90fe24872174077bb822061f9a5c0f2c80a043f11f99861cc1fd6e27"}
Oct 30 19:26:40.197742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.197724    5704 scope.go:122] "RemoveContainer" containerID="2bd5c17246c59f16d9135de9e71196240dd7d5383bc40ff204bc6dbd52780d71"
Oct 30 19:26:40.206693 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.206676    5704 scope.go:122] "RemoveContainer" containerID="2bd5c17246c59f16d9135de9e71196240dd7d5383bc40ff204bc6dbd52780d71"
Oct 30 19:26:40.206923 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:26:40.206902    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bd5c17246c59f16d9135de9e71196240dd7d5383bc40ff204bc6dbd52780d71\": container with ID starting with 2bd5c17246c59f16d9135de9e71196240dd7d5383bc40ff204bc6dbd52780d71 not found: ID does not exist" containerID="2bd5c17246c59f16d9135de9e71196240dd7d5383bc40ff204bc6dbd52780d71"
Oct 30 19:26:40.206984 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.206933    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bd5c17246c59f16d9135de9e71196240dd7d5383bc40ff204bc6dbd52780d71"} err="failed to get container status \"2bd5c17246c59f16d9135de9e71196240dd7d5383bc40ff204bc6dbd52780d71\": rpc error: code = NotFound desc = could not find container \"2bd5c17246c59f16d9135de9e71196240dd7d5383bc40ff204bc6dbd52780d71\": container with ID starting with 2bd5c17246c59f16d9135de9e71196240dd7d5383bc40ff204bc6dbd52780d71 not found: ID does not exist"
Oct 30 19:26:40.343857 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.343830    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"]
Oct 30 19:26:40.349651 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.349626    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-905566-87cb4cc67-7glpq"]
Oct 30 19:26:40.390238 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.390171    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:40.452358 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.452323    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"]
Oct 30 19:26:40.464427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:40.464395    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-324ebe-54c94bcfd5-hjmsh"]
Oct 30 19:26:41.180001 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:41.179970    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7cae896c-360e-492f-b987-dec361cf72e4" path="/var/lib/kubelet/pods/7cae896c-360e-492f-b987-dec361cf72e4/volumes"
Oct 30 19:26:41.180402 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:41.180388    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="95061afb-d0ef-47d6-975e-fc5d7d500e16" path="/var/lib/kubelet/pods/95061afb-d0ef-47d6-975e-fc5d7d500e16/volumes"
Oct 30 19:26:41.180777 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:41.180764    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c5c5d61a-a137-469c-8da5-3ceb9596ece2" path="/var/lib/kubelet/pods/c5c5d61a-a137-469c-8da5-3ceb9596ece2/volumes"
Oct 30 19:26:41.301697 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:41.301661    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:41.571256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:41.571222    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:41.750991 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:41.750960    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:42.299397 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:42.299360    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:46.714657 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:46.698422    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:48.787304 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:48.787272    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:49.019907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:49.019874    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:49.125215 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:49.125096    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"]
Oct 30 19:26:49.125356 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:49.125323    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy" containerID="cri-o://6973a05e427a9516818d575f83a23efda5188efea7dd6fd703c47fe0d572e530" gracePeriod=30
Oct 30 19:26:49.216944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:49.214529    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:51.043721 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:51.043684    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:51.044447 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:51.044424    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"]
Oct 30 19:26:51.044743 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:51.044685    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd" podUID="fda20ce9-bc4f-416d-8943-7fefd92f810a" containerName="istio-proxy" containerID="cri-o://9f12360dc808459b64c9137ab3d51a336229b1ac57042fbe70bf677debc43e45" gracePeriod=30
Oct 30 19:26:52.228459 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:52.228422    5704 patch_prober.go:29] interesting pod/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:26:52.228848 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:52.228486    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:26:52.534563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:52.534526    5704 patch_prober.go:29] interesting pod/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:26:52.534726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:52.534594    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd" podUID="fda20ce9-bc4f-416d-8943-7fefd92f810a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:26:53.066063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.063016    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"]
Oct 30 19:26:53.066063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.064561    5704 cpu_manager.go:414] "containerMap: removing container" podUID="7cae896c-360e-492f-b987-dec361cf72e4" containerName="maas-api"
Oct 30 19:26:53.066063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.064582    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7cae896c-360e-492f-b987-dec361cf72e4" containerName="maas-api"
Oct 30 19:26:53.066063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.064659    5704 cpu_manager.go:414] "containerMap: removing container" podUID="c5c5d61a-a137-469c-8da5-3ceb9596ece2" containerName="payload-pre-processing"
Oct 30 19:26:53.066063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.064669    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c5c5d61a-a137-469c-8da5-3ceb9596ece2" containerName="payload-pre-processing"
Oct 30 19:26:53.066063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.064775    5704 cpu_manager.go:414] "containerMap: removing container" podUID="95061afb-d0ef-47d6-975e-fc5d7d500e16" containerName="maas-api"
Oct 30 19:26:53.066063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.064784    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="95061afb-d0ef-47d6-975e-fc5d7d500e16" containerName="maas-api"
Oct 30 19:26:53.066063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.065106    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="c5c5d61a-a137-469c-8da5-3ceb9596ece2" containerName="payload-pre-processing"
Oct 30 19:26:53.066063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.065150    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="95061afb-d0ef-47d6-975e-fc5d7d500e16" containerName="maas-api"
Oct 30 19:26:53.066063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.065162    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="7cae896c-360e-492f-b987-dec361cf72e4" containerName="maas-api"
Oct 30 19:26:53.074587 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.074557    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.097683 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.097656    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-metrics-certs\") pod \"maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.097835 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.097757    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5g2z\" (UniqueName: \"kubernetes.io/projected/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-kube-api-access-x5g2z\") pod \"maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.097978 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.097955    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.098071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.098018    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-maas-api-tls\") pod \"maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.129644 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.129615    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"]
Oct 30 19:26:53.173015 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.172990    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-4833f3-serving-cert\""
Oct 30 19:26:53.199081 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.199045    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.199260 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.199101    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-maas-api-tls\") pod \"maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.199260 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.199170    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-metrics-certs\") pod \"maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.199388 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.199278    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5g2z\" (UniqueName: \"kubernetes.io/projected/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-kube-api-access-x5g2z\") pod \"maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.199388 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:26:53.199299    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-serving-cert: secret "maas-api-e2e-praxis-a-4833f3-serving-cert" not found
Oct 30 19:26:53.199388 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:26:53.199384    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-maas-api-tls podName:e1de5a60-9b29-46a8-a318-a3ff1eb48b32 nodeName:}" failed. No retries permitted until 2026-10-30 19:26:53.699362748 +0000 UTC m=+2386.645530656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-maas-api-tls") pod "maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw" (UID: "e1de5a60-9b29-46a8-a318-a3ff1eb48b32") : secret "maas-api-e2e-praxis-a-4833f3-serving-cert" not found
Oct 30 19:26:53.199894 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.199869    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.202730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.202704    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-metrics-certs\") pod \"maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.258888 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.258855    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5g2z\" (UniqueName: \"kubernetes.io/projected/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-kube-api-access-x5g2z\") pod \"maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.290465 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.290421    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:53.476262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.476169    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:53.662112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.662073    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:53.706626 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.706599    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-maas-api-tls\") pod \"maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.709116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.709094    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-maas-api-tls\") pod \"maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:53.988116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:53.988088    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:54.312874 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.312835    5704 generic.go:356] "Generic (PLEG): container finished" podID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerID="6973a05e427a9516818d575f83a23efda5188efea7dd6fd703c47fe0d572e530" exitCode=0
Oct 30 19:26:54.313347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.312873    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" event={"ID":"096b94a8-ca2d-419e-9ce4-3554b762cbc8","Type":"ContainerDied","Data":"6973a05e427a9516818d575f83a23efda5188efea7dd6fd703c47fe0d572e530"}
Oct 30 19:26:54.313347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.312943    5704 scope.go:122] "RemoveContainer" containerID="ed9bcd6f85b66884125389f3c3ba006b2ca67c240a175ef000383d338498e017"
Oct 30 19:26:54.359964 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.359937    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"]
Oct 30 19:26:54.404786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.404737    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:26:54.515922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.515896    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-token\" (UniqueName: \"kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-token\") pod \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") "
Oct 30 19:26:54.516091 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.515969    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-certs\") pod \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") "
Oct 30 19:26:54.516091 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.516073    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istiod-ca-cert\") pod \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") "
Oct 30 19:26:54.516213 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.516137    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-credential-socket\") pod \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") "
Oct 30 19:26:54.516275 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.516253    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-data\") pod \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") "
Oct 30 19:26:54.516339 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.516280    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-certs" pod "096b94a8-ca2d-419e-9ce4-3554b762cbc8" (UID: "096b94a8-ca2d-419e-9ce4-3554b762cbc8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:26:54.516405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.516368    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-credential-socket" pod "096b94a8-ca2d-419e-9ce4-3554b762cbc8" (UID: "096b94a8-ca2d-419e-9ce4-3554b762cbc8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:26:54.516405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.516386    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-envoy\") pod \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") "
Oct 30 19:26:54.516511 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.516437    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-socket\") pod \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") "
Oct 30 19:26:54.516511 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.516477    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-podinfo\") pod \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") "
Oct 30 19:26:54.516511 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.516484    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istiod-ca-cert" pod "096b94a8-ca2d-419e-9ce4-3554b762cbc8" (UID: "096b94a8-ca2d-419e-9ce4-3554b762cbc8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:26:54.516511 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.516491    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-data" pod "096b94a8-ca2d-419e-9ce4-3554b762cbc8" (UID: "096b94a8-ca2d-419e-9ce4-3554b762cbc8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:26:54.516722 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.516529    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-kube-api-access-rpghj\" (UniqueName: \"kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-kube-api-access-rpghj\") pod \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\" (UID: \"096b94a8-ca2d-419e-9ce4-3554b762cbc8\") "
Oct 30 19:26:54.516781 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.516761    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-socket" pod "096b94a8-ca2d-419e-9ce4-3554b762cbc8" (UID: "096b94a8-ca2d-419e-9ce4-3554b762cbc8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:26:54.517175 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.517148    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:54.517175 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.517177    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:54.517389 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.517191    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:54.517389 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.517205    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:54.517389 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.517218    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:54.518239 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.518213    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-token" pod "096b94a8-ca2d-419e-9ce4-3554b762cbc8" (UID: "096b94a8-ca2d-419e-9ce4-3554b762cbc8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:26:54.518576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.518557    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-podinfo" pod "096b94a8-ca2d-419e-9ce4-3554b762cbc8" (UID: "096b94a8-ca2d-419e-9ce4-3554b762cbc8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:26:54.518637 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.518593    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-envoy" pod "096b94a8-ca2d-419e-9ce4-3554b762cbc8" (UID: "096b94a8-ca2d-419e-9ce4-3554b762cbc8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:26:54.518708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.518694    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-kube-api-access-rpghj" pod "096b94a8-ca2d-419e-9ce4-3554b762cbc8" (UID: "096b94a8-ca2d-419e-9ce4-3554b762cbc8"). InnerVolumeSpecName "kube-api-access-rpghj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:26:54.618540 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.618510    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:54.618540 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.618535    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:54.618540 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.618544    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rpghj\" (UniqueName: \"kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-kube-api-access-rpghj\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:54.618764 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:54.618554    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/096b94a8-ca2d-419e-9ce4-3554b762cbc8-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:55.323069 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:55.323025    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw" event={"ID":"e1de5a60-9b29-46a8-a318-a3ff1eb48b32","Type":"ContainerStarted","Data":"d11ad2ba0b610e5cf33ce1df4d6a58cd0922cd07918355af12c1c35deb89d597"}
Oct 30 19:26:55.324418 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:55.324398    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"
Oct 30 19:26:55.324549 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:55.324422    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh" event={"ID":"096b94a8-ca2d-419e-9ce4-3554b762cbc8","Type":"ContainerDied","Data":"9ec0bfbb5148993491d73fed3cfacd7f51860e1631632b6dc9ceca0271389b5d"}
Oct 30 19:26:55.324549 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:55.324488    5704 scope.go:122] "RemoveContainer" containerID="6973a05e427a9516818d575f83a23efda5188efea7dd6fd703c47fe0d572e530"
Oct 30 19:26:55.521124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:55.521088    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"]
Oct 30 19:26:55.524774 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:55.524750    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-324ebe-openshift-default-648668947f-s7fkh"]
Oct 30 19:26:56.345217 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.345186    5704 generic.go:356] "Generic (PLEG): container finished" podID="fda20ce9-bc4f-416d-8943-7fefd92f810a" containerID="9f12360dc808459b64c9137ab3d51a336229b1ac57042fbe70bf677debc43e45" exitCode=0
Oct 30 19:26:56.345596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.345275    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd" event={"ID":"fda20ce9-bc4f-416d-8943-7fefd92f810a","Type":"ContainerDied","Data":"9f12360dc808459b64c9137ab3d51a336229b1ac57042fbe70bf677debc43e45"}
Oct 30 19:26:56.345596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.345328    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd" event={"ID":"fda20ce9-bc4f-416d-8943-7fefd92f810a","Type":"ContainerDied","Data":"b00919dbe518df94837ebdc91dec22a1ec72e03b7150b1dece69d1e93cce96f5"}
Oct 30 19:26:56.345596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.345351    5704 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b00919dbe518df94837ebdc91dec22a1ec72e03b7150b1dece69d1e93cce96f5"
Oct 30 19:26:56.346974 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.346948    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw" event={"ID":"e1de5a60-9b29-46a8-a318-a3ff1eb48b32","Type":"ContainerStarted","Data":"a9909badd1b3bcb363a4f9af6719b006571c48fd13e62f72edef1a73ab73c2eb"}
Oct 30 19:26:56.347216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.347068    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:26:56.347314 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.347260    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:26:56.440160 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.440131    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-kube-api-access-s8t8w\" (UniqueName: \"kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-kube-api-access-s8t8w\") pod \"fda20ce9-bc4f-416d-8943-7fefd92f810a\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") "
Oct 30 19:26:56.440332 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.440207    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-envoy\") pod \"fda20ce9-bc4f-416d-8943-7fefd92f810a\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") "
Oct 30 19:26:56.440332 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.440308    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-certs\") pod \"fda20ce9-bc4f-416d-8943-7fefd92f810a\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") "
Oct 30 19:26:56.440414 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.440375    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-token\" (UniqueName: \"kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-token\") pod \"fda20ce9-bc4f-416d-8943-7fefd92f810a\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") "
Oct 30 19:26:56.440492 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.440476    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-credential-socket\") pod \"fda20ce9-bc4f-416d-8943-7fefd92f810a\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") "
Oct 30 19:26:56.440542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.440514    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fda20ce9-bc4f-416d-8943-7fefd92f810a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fda20ce9-bc4f-416d-8943-7fefd92f810a-istiod-ca-cert\") pod \"fda20ce9-bc4f-416d-8943-7fefd92f810a\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") "
Oct 30 19:26:56.440613 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.440545    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-podinfo\") pod \"fda20ce9-bc4f-416d-8943-7fefd92f810a\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") "
Oct 30 19:26:56.440613 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.440563    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-certs" pod "fda20ce9-bc4f-416d-8943-7fefd92f810a" (UID: "fda20ce9-bc4f-416d-8943-7fefd92f810a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:26:56.440718 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.440622    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-socket\") pod \"fda20ce9-bc4f-416d-8943-7fefd92f810a\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") "
Oct 30 19:26:56.440718 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.440663    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-data\") pod \"fda20ce9-bc4f-416d-8943-7fefd92f810a\" (UID: \"fda20ce9-bc4f-416d-8943-7fefd92f810a\") "
Oct 30 19:26:56.440831 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.440716    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-credential-socket" pod "fda20ce9-bc4f-416d-8943-7fefd92f810a" (UID: "fda20ce9-bc4f-416d-8943-7fefd92f810a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:26:56.440924 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.440903    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fda20ce9-bc4f-416d-8943-7fefd92f810a-istiod-ca-cert" pod "fda20ce9-bc4f-416d-8943-7fefd92f810a" (UID: "fda20ce9-bc4f-416d-8943-7fefd92f810a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:26:56.440982 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.440933    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-socket" pod "fda20ce9-bc4f-416d-8943-7fefd92f810a" (UID: "fda20ce9-bc4f-416d-8943-7fefd92f810a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:26:56.441165 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.441145    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-data" pod "fda20ce9-bc4f-416d-8943-7fefd92f810a" (UID: "fda20ce9-bc4f-416d-8943-7fefd92f810a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:26:56.441442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.441388    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:56.441442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.441416    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fda20ce9-bc4f-416d-8943-7fefd92f810a-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:56.441442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.441432    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:56.441647 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.441446    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:56.441647 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.441460    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:56.442781 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.442757    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-kube-api-access-s8t8w" pod "fda20ce9-bc4f-416d-8943-7fefd92f810a" (UID: "fda20ce9-bc4f-416d-8943-7fefd92f810a"). InnerVolumeSpecName "kube-api-access-s8t8w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:26:56.442902 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.442888    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-envoy" pod "fda20ce9-bc4f-416d-8943-7fefd92f810a" (UID: "fda20ce9-bc4f-416d-8943-7fefd92f810a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:26:56.442960 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.442902    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-token" pod "fda20ce9-bc4f-416d-8943-7fefd92f810a" (UID: "fda20ce9-bc4f-416d-8943-7fefd92f810a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:26:56.442960 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.442909    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-podinfo" pod "fda20ce9-bc4f-416d-8943-7fefd92f810a" (UID: "fda20ce9-bc4f-416d-8943-7fefd92f810a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:26:56.459666 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.459611    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw" podStartSLOduration=2.573290162 podStartE2EDuration="3.459593558s" totalImagesPullingTime="886.303396ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:26:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:26:56.457649817 +0000 UTC m=+2389.403817747" watchObservedRunningTime="2026-10-30 19:26:56.459593558 +0000 UTC m=+2389.405761465"
Oct 30 19:26:56.543024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.542995    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8t8w\" (UniqueName: \"kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-kube-api-access-s8t8w\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:56.543024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.543026    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:56.543024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.543054    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:56.543282 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:56.543067    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fda20ce9-bc4f-416d-8943-7fefd92f810a-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:26:57.180768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:57.180738    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" path="/var/lib/kubelet/pods/096b94a8-ca2d-419e-9ce4-3554b762cbc8/volumes"
Oct 30 19:26:57.353921 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:57.353891    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"
Oct 30 19:26:57.425779 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:57.425745    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:57.512436 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:57.512390    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"]
Oct 30 19:26:57.518332 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:57.518296    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-9gzpd"]
Oct 30 19:26:57.671016 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:57.670975    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:58.385908 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:58.385856    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:26:59.180896 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:59.180846    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fda20ce9-bc4f-416d-8943-7fefd92f810a" path="/var/lib/kubelet/pods/fda20ce9-bc4f-416d-8943-7fefd92f810a/volumes"
Oct 30 19:26:59.713464 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:26:59.713426    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:27:01.485954 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:01.485920    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"]
Oct 30 19:27:01.755441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:01.755404    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:01.934524 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:01.934474    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:02.359586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:02.359555    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:27:03.117900 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:03.117865    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:03.383530 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:27:03.383450    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:27:06.487375 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:06.487338    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps" podUID="46f7284e-cdd6-49c8-9f65-daff0f30f8bb" containerName="payload-pre-processing" containerID="cri-o://5eca2a6f138d93f20f2d624ebe22754026175b5e646270b8b338d259c67d3c26" gracePeriod=25
Oct 30 19:27:06.753602 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:06.753578    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"
Oct 30 19:27:06.881174 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:06.881147    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-plugins-config-volume\") pod \"46f7284e-cdd6-49c8-9f65-daff0f30f8bb\" (UID: \"46f7284e-cdd6-49c8-9f65-daff0f30f8bb\") "
Oct 30 19:27:06.881368 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:06.881189    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-kube-api-access-mtmqd\" (UniqueName: \"kubernetes.io/projected/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-kube-api-access-mtmqd\") pod \"46f7284e-cdd6-49c8-9f65-daff0f30f8bb\" (UID: \"46f7284e-cdd6-49c8-9f65-daff0f30f8bb\") "
Oct 30 19:27:06.881603 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:06.881568    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-plugins-config-volume" pod "46f7284e-cdd6-49c8-9f65-daff0f30f8bb" (UID: "46f7284e-cdd6-49c8-9f65-daff0f30f8bb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:06.883362 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:06.883334    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-kube-api-access-mtmqd" pod "46f7284e-cdd6-49c8-9f65-daff0f30f8bb" (UID: "46f7284e-cdd6-49c8-9f65-daff0f30f8bb"). InnerVolumeSpecName "kube-api-access-mtmqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:06.982443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:06.982416    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:06.982443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:06.982440    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mtmqd\" (UniqueName: \"kubernetes.io/projected/46f7284e-cdd6-49c8-9f65-daff0f30f8bb-kube-api-access-mtmqd\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:07.182845 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.182768    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:07.430170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.430020    5704 generic.go:356] "Generic (PLEG): container finished" podID="46f7284e-cdd6-49c8-9f65-daff0f30f8bb" containerID="5eca2a6f138d93f20f2d624ebe22754026175b5e646270b8b338d259c67d3c26" exitCode=0
Oct 30 19:27:07.430701 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.430661    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps" event={"ID":"46f7284e-cdd6-49c8-9f65-daff0f30f8bb","Type":"ContainerDied","Data":"5eca2a6f138d93f20f2d624ebe22754026175b5e646270b8b338d259c67d3c26"}
Oct 30 19:27:07.442791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.431061    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps" event={"ID":"46f7284e-cdd6-49c8-9f65-daff0f30f8bb","Type":"ContainerDied","Data":"33a0c6f4bb61ac8e7602d43b02a8fef90aa6d7e02bfae7f2db2f1a681b34b298"}
Oct 30 19:27:07.443145 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.443118    5704 scope.go:122] "RemoveContainer" containerID="5eca2a6f138d93f20f2d624ebe22754026175b5e646270b8b338d259c67d3c26"
Oct 30 19:27:07.443545 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.443528    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"
Oct 30 19:27:07.487935 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.487908    5704 scope.go:122] "RemoveContainer" containerID="5eca2a6f138d93f20f2d624ebe22754026175b5e646270b8b338d259c67d3c26"
Oct 30 19:27:07.488393 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:27:07.488365    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5eca2a6f138d93f20f2d624ebe22754026175b5e646270b8b338d259c67d3c26\": container with ID starting with 5eca2a6f138d93f20f2d624ebe22754026175b5e646270b8b338d259c67d3c26 not found: ID does not exist" containerID="5eca2a6f138d93f20f2d624ebe22754026175b5e646270b8b338d259c67d3c26"
Oct 30 19:27:07.488485 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.488407    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5eca2a6f138d93f20f2d624ebe22754026175b5e646270b8b338d259c67d3c26"} err="failed to get container status \"5eca2a6f138d93f20f2d624ebe22754026175b5e646270b8b338d259c67d3c26\": rpc error: code = NotFound desc = could not find container \"5eca2a6f138d93f20f2d624ebe22754026175b5e646270b8b338d259c67d3c26\": container with ID starting with 5eca2a6f138d93f20f2d624ebe22754026175b5e646270b8b338d259c67d3c26 not found: ID does not exist"
Oct 30 19:27:07.497844 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.497811    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"]
Oct 30 19:27:07.498022 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.498003    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn" podUID="d0289572-5d8d-4e2f-a14d-4bd58d050bd7" containerName="payload-processing" containerID="cri-o://947c5e86d15d7b492ee32374dcf46944ead98dc7606315409f48d58f41c03051" gracePeriod=30
Oct 30 19:27:07.549767 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.549740    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"]
Oct 30 19:27:07.549994 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.549955    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9" podUID="353a50b4-be00-45b2-a6b1-af6498c0af69" containerName="payload-pre-processing" containerID="cri-o://8d46d1e4dccf6358a17e541be48b88dc3b2418ec6ac699e4fcebebe311b25a11" gracePeriod=30
Oct 30 19:27:07.606535 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.606477    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"]
Oct 30 19:27:07.609957 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.609813    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-d6b5ac-96f7cc84c-z6tps"]
Oct 30 19:27:07.699105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.698886    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:07.850226 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.850200    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"
Oct 30 19:27:07.878067 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.878025    5704 scope.go:122] "RemoveContainer" containerID="9f12360dc808459b64c9137ab3d51a336229b1ac57042fbe70bf677debc43e45"
Oct 30 19:27:07.878426 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.878403    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"
Oct 30 19:27:07.883186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:07.883161    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:08.006233 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.006189    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-kube-api-access-545pp\" (UniqueName: \"kubernetes.io/projected/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-kube-api-access-545pp\") pod \"d0289572-5d8d-4e2f-a14d-4bd58d050bd7\" (UID: \"d0289572-5d8d-4e2f-a14d-4bd58d050bd7\") "
Oct 30 19:27:08.006422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.006360    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/353a50b4-be00-45b2-a6b1-af6498c0af69-kube-api-access-bnv5c\" (UniqueName: \"kubernetes.io/projected/353a50b4-be00-45b2-a6b1-af6498c0af69-kube-api-access-bnv5c\") pod \"353a50b4-be00-45b2-a6b1-af6498c0af69\" (UID: \"353a50b4-be00-45b2-a6b1-af6498c0af69\") "
Oct 30 19:27:08.006503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.006437    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-plugins-config-volume\") pod \"d0289572-5d8d-4e2f-a14d-4bd58d050bd7\" (UID: \"d0289572-5d8d-4e2f-a14d-4bd58d050bd7\") "
Oct 30 19:27:08.006503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.006493    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/353a50b4-be00-45b2-a6b1-af6498c0af69-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/353a50b4-be00-45b2-a6b1-af6498c0af69-plugins-config-volume\") pod \"353a50b4-be00-45b2-a6b1-af6498c0af69\" (UID: \"353a50b4-be00-45b2-a6b1-af6498c0af69\") "
Oct 30 19:27:08.008878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.007466    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/353a50b4-be00-45b2-a6b1-af6498c0af69-plugins-config-volume" pod "353a50b4-be00-45b2-a6b1-af6498c0af69" (UID: "353a50b4-be00-45b2-a6b1-af6498c0af69"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:08.014410 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.014271    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-plugins-config-volume" pod "d0289572-5d8d-4e2f-a14d-4bd58d050bd7" (UID: "d0289572-5d8d-4e2f-a14d-4bd58d050bd7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:08.023072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.022469    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/353a50b4-be00-45b2-a6b1-af6498c0af69-kube-api-access-bnv5c" pod "353a50b4-be00-45b2-a6b1-af6498c0af69" (UID: "353a50b4-be00-45b2-a6b1-af6498c0af69"). InnerVolumeSpecName "kube-api-access-bnv5c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:08.024931 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.024863    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-kube-api-access-545pp" pod "d0289572-5d8d-4e2f-a14d-4bd58d050bd7" (UID: "d0289572-5d8d-4e2f-a14d-4bd58d050bd7"). InnerVolumeSpecName "kube-api-access-545pp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:08.108719 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.108688    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:08.108904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.108736    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/353a50b4-be00-45b2-a6b1-af6498c0af69-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:08.108904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.108748    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-545pp\" (UniqueName: \"kubernetes.io/projected/d0289572-5d8d-4e2f-a14d-4bd58d050bd7-kube-api-access-545pp\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:08.108904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.108757    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bnv5c\" (UniqueName: \"kubernetes.io/projected/353a50b4-be00-45b2-a6b1-af6498c0af69-kube-api-access-bnv5c\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:08.437633 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.437546    5704 generic.go:356] "Generic (PLEG): container finished" podID="353a50b4-be00-45b2-a6b1-af6498c0af69" containerID="8d46d1e4dccf6358a17e541be48b88dc3b2418ec6ac699e4fcebebe311b25a11" exitCode=0
Oct 30 19:27:08.437633 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.437594    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"
Oct 30 19:27:08.437865 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.437630    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9" event={"ID":"353a50b4-be00-45b2-a6b1-af6498c0af69","Type":"ContainerDied","Data":"8d46d1e4dccf6358a17e541be48b88dc3b2418ec6ac699e4fcebebe311b25a11"}
Oct 30 19:27:08.437865 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.437694    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9" event={"ID":"353a50b4-be00-45b2-a6b1-af6498c0af69","Type":"ContainerDied","Data":"59c116447cc6ed0dc98afa7e9c1774e975c18c9dbc79c723b6ff6083ce0af672"}
Oct 30 19:27:08.437865 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.437731    5704 scope.go:122] "RemoveContainer" containerID="8d46d1e4dccf6358a17e541be48b88dc3b2418ec6ac699e4fcebebe311b25a11"
Oct 30 19:27:08.439178 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.439141    5704 generic.go:356] "Generic (PLEG): container finished" podID="d0289572-5d8d-4e2f-a14d-4bd58d050bd7" containerID="947c5e86d15d7b492ee32374dcf46944ead98dc7606315409f48d58f41c03051" exitCode=0
Oct 30 19:27:08.439285 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.439194    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"
Oct 30 19:27:08.439285 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.439237    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn" event={"ID":"d0289572-5d8d-4e2f-a14d-4bd58d050bd7","Type":"ContainerDied","Data":"947c5e86d15d7b492ee32374dcf46944ead98dc7606315409f48d58f41c03051"}
Oct 30 19:27:08.439371 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.439286    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn" event={"ID":"d0289572-5d8d-4e2f-a14d-4bd58d050bd7","Type":"ContainerDied","Data":"02177cf2e9d39532b696e341684c86c014b2bb1dee8a4ba2fd11263f53aeff2d"}
Oct 30 19:27:08.449317 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.449301    5704 scope.go:122] "RemoveContainer" containerID="8d46d1e4dccf6358a17e541be48b88dc3b2418ec6ac699e4fcebebe311b25a11"
Oct 30 19:27:08.449570 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:27:08.449553    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d46d1e4dccf6358a17e541be48b88dc3b2418ec6ac699e4fcebebe311b25a11\": container with ID starting with 8d46d1e4dccf6358a17e541be48b88dc3b2418ec6ac699e4fcebebe311b25a11 not found: ID does not exist" containerID="8d46d1e4dccf6358a17e541be48b88dc3b2418ec6ac699e4fcebebe311b25a11"
Oct 30 19:27:08.449627 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.449576    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d46d1e4dccf6358a17e541be48b88dc3b2418ec6ac699e4fcebebe311b25a11"} err="failed to get container status \"8d46d1e4dccf6358a17e541be48b88dc3b2418ec6ac699e4fcebebe311b25a11\": rpc error: code = NotFound desc = could not find container \"8d46d1e4dccf6358a17e541be48b88dc3b2418ec6ac699e4fcebebe311b25a11\": container with ID starting with 8d46d1e4dccf6358a17e541be48b88dc3b2418ec6ac699e4fcebebe311b25a11 not found: ID does not exist"
Oct 30 19:27:08.449627 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.449591    5704 scope.go:122] "RemoveContainer" containerID="947c5e86d15d7b492ee32374dcf46944ead98dc7606315409f48d58f41c03051"
Oct 30 19:27:08.458069 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.458025    5704 scope.go:122] "RemoveContainer" containerID="947c5e86d15d7b492ee32374dcf46944ead98dc7606315409f48d58f41c03051"
Oct 30 19:27:08.458300 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:27:08.458283    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"947c5e86d15d7b492ee32374dcf46944ead98dc7606315409f48d58f41c03051\": container with ID starting with 947c5e86d15d7b492ee32374dcf46944ead98dc7606315409f48d58f41c03051 not found: ID does not exist" containerID="947c5e86d15d7b492ee32374dcf46944ead98dc7606315409f48d58f41c03051"
Oct 30 19:27:08.458351 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.458305    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"947c5e86d15d7b492ee32374dcf46944ead98dc7606315409f48d58f41c03051"} err="failed to get container status \"947c5e86d15d7b492ee32374dcf46944ead98dc7606315409f48d58f41c03051\": rpc error: code = NotFound desc = could not find container \"947c5e86d15d7b492ee32374dcf46944ead98dc7606315409f48d58f41c03051\": container with ID starting with 947c5e86d15d7b492ee32374dcf46944ead98dc7606315409f48d58f41c03051 not found: ID does not exist"
Oct 30 19:27:08.592148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.592121    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"]
Oct 30 19:27:08.596607 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.596587    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-4ec347-7b48cd9f9d-sldfn"]
Oct 30 19:27:08.698103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.698023    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"]
Oct 30 19:27:08.704358 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.704333    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-4ec347-76f7d564d7-sscg9"]
Oct 30 19:27:08.711288 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.711264    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:08.920515 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:08.920484    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:09.101564 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:09.101522    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:09.179543 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:09.179512    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="353a50b4-be00-45b2-a6b1-af6498c0af69" path="/var/lib/kubelet/pods/353a50b4-be00-45b2-a6b1-af6498c0af69/volumes"
Oct 30 19:27:09.179929 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:09.179917    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="46f7284e-cdd6-49c8-9f65-daff0f30f8bb" path="/var/lib/kubelet/pods/46f7284e-cdd6-49c8-9f65-daff0f30f8bb/volumes"
Oct 30 19:27:09.180307 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:09.180293    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d0289572-5d8d-4e2f-a14d-4bd58d050bd7" path="/var/lib/kubelet/pods/d0289572-5d8d-4e2f-a14d-4bd58d050bd7/volumes"
Oct 30 19:27:13.982057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:13.982013    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:19.658848 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:19.658812    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:20.397944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:20.397911    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:25.483220 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:25.483173    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:26.329342 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:26.329307    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:26.593452 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:26.593351    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:26.680864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:26.680827    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"]
Oct 30 19:27:26.681080 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:26.681058    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf" podUID="793048dc-32db-41ad-9964-04c275d02cbf" containerName="istio-proxy" containerID="cri-o://3072ecd263b3d61aec49aebf52648071def1d44f424ba46ed9795ef2af7b4727" gracePeriod=30
Oct 30 19:27:26.786851 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:26.786820    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:30.793063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:30.793001    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:31.158058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:31.157947    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"]
Oct 30 19:27:31.356353 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:31.356317    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:31.542586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:31.542548    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:31.726145 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:31.726119    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:31.946781 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:31.946760    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:27:32.084262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084178    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-credential-socket\") pod \"793048dc-32db-41ad-9964-04c275d02cbf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") "
Oct 30 19:27:32.084262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084229    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-certs\") pod \"793048dc-32db-41ad-9964-04c275d02cbf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") "
Oct 30 19:27:32.084262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084250    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-data\") pod \"793048dc-32db-41ad-9964-04c275d02cbf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") "
Oct 30 19:27:32.084551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084278    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/793048dc-32db-41ad-9964-04c275d02cbf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/793048dc-32db-41ad-9964-04c275d02cbf-istiod-ca-cert\") pod \"793048dc-32db-41ad-9964-04c275d02cbf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") "
Oct 30 19:27:32.084551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084304    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-istio-token\" (UniqueName: \"kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-istio-token\") pod \"793048dc-32db-41ad-9964-04c275d02cbf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") "
Oct 30 19:27:32.084551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084447    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-envoy\") pod \"793048dc-32db-41ad-9964-04c275d02cbf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") "
Oct 30 19:27:32.084551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084455    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-credential-socket" pod "793048dc-32db-41ad-9964-04c275d02cbf" (UID: "793048dc-32db-41ad-9964-04c275d02cbf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:32.084768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084529    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-certs" pod "793048dc-32db-41ad-9964-04c275d02cbf" (UID: "793048dc-32db-41ad-9964-04c275d02cbf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:32.084768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084539    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/793048dc-32db-41ad-9964-04c275d02cbf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/793048dc-32db-41ad-9964-04c275d02cbf-istio-podinfo\") pod \"793048dc-32db-41ad-9964-04c275d02cbf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") "
Oct 30 19:27:32.084768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084628    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-data" pod "793048dc-32db-41ad-9964-04c275d02cbf" (UID: "793048dc-32db-41ad-9964-04c275d02cbf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:32.084768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084635    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-socket\") pod \"793048dc-32db-41ad-9964-04c275d02cbf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") "
Oct 30 19:27:32.084768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084721    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-kube-api-access-kstrr\" (UniqueName: \"kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-kube-api-access-kstrr\") pod \"793048dc-32db-41ad-9964-04c275d02cbf\" (UID: \"793048dc-32db-41ad-9964-04c275d02cbf\") "
Oct 30 19:27:32.084768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084729    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/793048dc-32db-41ad-9964-04c275d02cbf-istiod-ca-cert" pod "793048dc-32db-41ad-9964-04c275d02cbf" (UID: "793048dc-32db-41ad-9964-04c275d02cbf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:32.085121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.084827    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-socket" pod "793048dc-32db-41ad-9964-04c275d02cbf" (UID: "793048dc-32db-41ad-9964-04c275d02cbf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:32.085314 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.085284    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:32.085314 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.085309    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:32.085460 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.085324    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:32.085460 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.085340    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/793048dc-32db-41ad-9964-04c275d02cbf-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:32.085460 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.085354    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:32.087006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.086980    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-istio-token" pod "793048dc-32db-41ad-9964-04c275d02cbf" (UID: "793048dc-32db-41ad-9964-04c275d02cbf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:32.087192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.087169    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-envoy" pod "793048dc-32db-41ad-9964-04c275d02cbf" (UID: "793048dc-32db-41ad-9964-04c275d02cbf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:27:32.087293 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.087174    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/793048dc-32db-41ad-9964-04c275d02cbf-istio-podinfo" pod "793048dc-32db-41ad-9964-04c275d02cbf" (UID: "793048dc-32db-41ad-9964-04c275d02cbf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:27:32.087363 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.087329    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-kube-api-access-kstrr" pod "793048dc-32db-41ad-9964-04c275d02cbf" (UID: "793048dc-32db-41ad-9964-04c275d02cbf"). InnerVolumeSpecName "kube-api-access-kstrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:32.186302 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.186279    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:32.186302 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.186302    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/793048dc-32db-41ad-9964-04c275d02cbf-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:32.186473 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.186313    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/793048dc-32db-41ad-9964-04c275d02cbf-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:32.186473 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.186322    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kstrr\" (UniqueName: \"kubernetes.io/projected/793048dc-32db-41ad-9964-04c275d02cbf-kube-api-access-kstrr\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:32.635911 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.635875    5704 generic.go:356] "Generic (PLEG): container finished" podID="793048dc-32db-41ad-9964-04c275d02cbf" containerID="3072ecd263b3d61aec49aebf52648071def1d44f424ba46ed9795ef2af7b4727" exitCode=0
Oct 30 19:27:32.639218 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.635934    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf" event={"ID":"793048dc-32db-41ad-9964-04c275d02cbf","Type":"ContainerDied","Data":"3072ecd263b3d61aec49aebf52648071def1d44f424ba46ed9795ef2af7b4727"}
Oct 30 19:27:32.639218 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.636325    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf" event={"ID":"793048dc-32db-41ad-9964-04c275d02cbf","Type":"ContainerDied","Data":"cc041c28fbf0e1f767ccd4ba45f05847317cef72331a7711a34978e7e8ce5838"}
Oct 30 19:27:32.639218 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.636363    5704 scope.go:122] "RemoveContainer" containerID="3072ecd263b3d61aec49aebf52648071def1d44f424ba46ed9795ef2af7b4727"
Oct 30 19:27:32.639218 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.635971    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"
Oct 30 19:27:32.651292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.651274    5704 scope.go:122] "RemoveContainer" containerID="8dfe37d6425983618fd225a98a8621120318032baaccc38644649cee998d5ecb"
Oct 30 19:27:32.660321 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.660304    5704 scope.go:122] "RemoveContainer" containerID="3072ecd263b3d61aec49aebf52648071def1d44f424ba46ed9795ef2af7b4727"
Oct 30 19:27:32.660561 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:27:32.660545    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3072ecd263b3d61aec49aebf52648071def1d44f424ba46ed9795ef2af7b4727\": container with ID starting with 3072ecd263b3d61aec49aebf52648071def1d44f424ba46ed9795ef2af7b4727 not found: ID does not exist" containerID="3072ecd263b3d61aec49aebf52648071def1d44f424ba46ed9795ef2af7b4727"
Oct 30 19:27:32.660610 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.660569    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3072ecd263b3d61aec49aebf52648071def1d44f424ba46ed9795ef2af7b4727"} err="failed to get container status \"3072ecd263b3d61aec49aebf52648071def1d44f424ba46ed9795ef2af7b4727\": rpc error: code = NotFound desc = could not find container \"3072ecd263b3d61aec49aebf52648071def1d44f424ba46ed9795ef2af7b4727\": container with ID starting with 3072ecd263b3d61aec49aebf52648071def1d44f424ba46ed9795ef2af7b4727 not found: ID does not exist"
Oct 30 19:27:32.660610 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.660585    5704 scope.go:122] "RemoveContainer" containerID="8dfe37d6425983618fd225a98a8621120318032baaccc38644649cee998d5ecb"
Oct 30 19:27:32.660798 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:27:32.660782    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dfe37d6425983618fd225a98a8621120318032baaccc38644649cee998d5ecb\": container with ID starting with 8dfe37d6425983618fd225a98a8621120318032baaccc38644649cee998d5ecb not found: ID does not exist" containerID="8dfe37d6425983618fd225a98a8621120318032baaccc38644649cee998d5ecb"
Oct 30 19:27:32.660844 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.660801    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dfe37d6425983618fd225a98a8621120318032baaccc38644649cee998d5ecb"} err="failed to get container status \"8dfe37d6425983618fd225a98a8621120318032baaccc38644649cee998d5ecb\": rpc error: code = NotFound desc = could not find container \"8dfe37d6425983618fd225a98a8621120318032baaccc38644649cee998d5ecb\": container with ID starting with 8dfe37d6425983618fd225a98a8621120318032baaccc38644649cee998d5ecb not found: ID does not exist"
Oct 30 19:27:32.797295 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.797263    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"]
Oct 30 19:27:32.843100 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:32.843075    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-4ec347-openshift-default-6c5bbf9475-8l7mf"]
Oct 30 19:27:33.180373 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:33.180345    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="793048dc-32db-41ad-9964-04c275d02cbf" path="/var/lib/kubelet/pods/793048dc-32db-41ad-9964-04c275d02cbf/volumes"
Oct 30 19:27:33.383461 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:27:33.383423    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:27:36.158479 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.158423    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm" podUID="a37355ab-b5c4-48af-b492-6536e09680dc" containerName="payload-pre-processing" containerID="cri-o://b70a803f0b1a054af53f9612bcce36ca3377d9eeacf33a6a8981746a9613c65d" gracePeriod=25
Oct 30 19:27:36.421350 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.421297    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"
Oct 30 19:27:36.530949 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.530879    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a37355ab-b5c4-48af-b492-6536e09680dc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a37355ab-b5c4-48af-b492-6536e09680dc-plugins-config-volume\") pod \"a37355ab-b5c4-48af-b492-6536e09680dc\" (UID: \"a37355ab-b5c4-48af-b492-6536e09680dc\") "
Oct 30 19:27:36.531143 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.531009    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a37355ab-b5c4-48af-b492-6536e09680dc-kube-api-access-qz6qk\" (UniqueName: \"kubernetes.io/projected/a37355ab-b5c4-48af-b492-6536e09680dc-kube-api-access-qz6qk\") pod \"a37355ab-b5c4-48af-b492-6536e09680dc\" (UID: \"a37355ab-b5c4-48af-b492-6536e09680dc\") "
Oct 30 19:27:36.531237 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.531215    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a37355ab-b5c4-48af-b492-6536e09680dc-plugins-config-volume" pod "a37355ab-b5c4-48af-b492-6536e09680dc" (UID: "a37355ab-b5c4-48af-b492-6536e09680dc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:27:36.531569 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.531549    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a37355ab-b5c4-48af-b492-6536e09680dc-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:36.533095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.533073    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a37355ab-b5c4-48af-b492-6536e09680dc-kube-api-access-qz6qk" pod "a37355ab-b5c4-48af-b492-6536e09680dc" (UID: "a37355ab-b5c4-48af-b492-6536e09680dc"). InnerVolumeSpecName "kube-api-access-qz6qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:27:36.633183 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.633156    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qz6qk\" (UniqueName: \"kubernetes.io/projected/a37355ab-b5c4-48af-b492-6536e09680dc-kube-api-access-qz6qk\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:27:36.671740 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.671670    5704 generic.go:356] "Generic (PLEG): container finished" podID="a37355ab-b5c4-48af-b492-6536e09680dc" containerID="b70a803f0b1a054af53f9612bcce36ca3377d9eeacf33a6a8981746a9613c65d" exitCode=0
Oct 30 19:27:36.671740 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.671717    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"
Oct 30 19:27:36.671922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.671751    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm" event={"ID":"a37355ab-b5c4-48af-b492-6536e09680dc","Type":"ContainerDied","Data":"b70a803f0b1a054af53f9612bcce36ca3377d9eeacf33a6a8981746a9613c65d"}
Oct 30 19:27:36.671922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.671800    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm" event={"ID":"a37355ab-b5c4-48af-b492-6536e09680dc","Type":"ContainerDied","Data":"ba7d2e35f55d960dd3d1599b9c3eb016a90a5047b1ebb80c69ccfa7b1daa5874"}
Oct 30 19:27:36.671922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.671821    5704 scope.go:122] "RemoveContainer" containerID="b70a803f0b1a054af53f9612bcce36ca3377d9eeacf33a6a8981746a9613c65d"
Oct 30 19:27:36.681888 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.681872    5704 scope.go:122] "RemoveContainer" containerID="b70a803f0b1a054af53f9612bcce36ca3377d9eeacf33a6a8981746a9613c65d"
Oct 30 19:27:36.682160 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:27:36.682141    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b70a803f0b1a054af53f9612bcce36ca3377d9eeacf33a6a8981746a9613c65d\": container with ID starting with b70a803f0b1a054af53f9612bcce36ca3377d9eeacf33a6a8981746a9613c65d not found: ID does not exist" containerID="b70a803f0b1a054af53f9612bcce36ca3377d9eeacf33a6a8981746a9613c65d"
Oct 30 19:27:36.682213 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.682166    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b70a803f0b1a054af53f9612bcce36ca3377d9eeacf33a6a8981746a9613c65d"} err="failed to get container status \"b70a803f0b1a054af53f9612bcce36ca3377d9eeacf33a6a8981746a9613c65d\": rpc error: code = NotFound desc = could not find container \"b70a803f0b1a054af53f9612bcce36ca3377d9eeacf33a6a8981746a9613c65d\": container with ID starting with b70a803f0b1a054af53f9612bcce36ca3377d9eeacf33a6a8981746a9613c65d not found: ID does not exist"
Oct 30 19:27:36.827122 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.827095    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"]
Oct 30 19:27:36.832541 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:36.832518    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-85cc6567c9-x5qwm"]
Oct 30 19:27:37.179242 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:37.179209    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a37355ab-b5c4-48af-b492-6536e09680dc" path="/var/lib/kubelet/pods/a37355ab-b5c4-48af-b492-6536e09680dc/volumes"
Oct 30 19:27:37.215628 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:37.215589    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:38.816644 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:38.816612    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:27:42.266384 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:42.266339    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:47.608535 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:47.608499    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:53.864080 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:53.864015    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:27:58.752084 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:27:58.752023    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:01.038598 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:01.038558    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:02.438963 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:02.438925    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:03.208900 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:03.208699    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:03.384332 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:28:03.384298    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:28:03.394554 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:03.394519    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:03.721521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:03.721491    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:07.838470 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:07.838434    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:08.022677 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:08.022640    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:08.204516 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:08.204430    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:08.585672 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:08.585641    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:10.990119 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:10.990090    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:11.220635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.220598    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:11.329594 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.329559    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"]
Oct 30 19:28:11.330133 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330119    5704 cpu_manager.go:414] "containerMap: removing container" podUID="d0289572-5d8d-4e2f-a14d-4bd58d050bd7" containerName="payload-processing"
Oct 30 19:28:11.330185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330134    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d0289572-5d8d-4e2f-a14d-4bd58d050bd7" containerName="payload-processing"
Oct 30 19:28:11.330185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330151    5704 cpu_manager.go:414] "containerMap: removing container" podUID="fda20ce9-bc4f-416d-8943-7fefd92f810a" containerName="istio-proxy"
Oct 30 19:28:11.330185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330157    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fda20ce9-bc4f-416d-8943-7fefd92f810a" containerName="istio-proxy"
Oct 30 19:28:11.330185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330164    5704 cpu_manager.go:414] "containerMap: removing container" podUID="793048dc-32db-41ad-9964-04c275d02cbf" containerName="istio-proxy"
Oct 30 19:28:11.330185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330170    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="793048dc-32db-41ad-9964-04c275d02cbf" containerName="istio-proxy"
Oct 30 19:28:11.330185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330180    5704 cpu_manager.go:414] "containerMap: removing container" podUID="a37355ab-b5c4-48af-b492-6536e09680dc" containerName="payload-pre-processing"
Oct 30 19:28:11.330185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330186    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a37355ab-b5c4-48af-b492-6536e09680dc" containerName="payload-pre-processing"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330198    5704 cpu_manager.go:414] "containerMap: removing container" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330203    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330211    5704 cpu_manager.go:414] "containerMap: removing container" podUID="46f7284e-cdd6-49c8-9f65-daff0f30f8bb" containerName="payload-pre-processing"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330216    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="46f7284e-cdd6-49c8-9f65-daff0f30f8bb" containerName="payload-pre-processing"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330223    5704 cpu_manager.go:414] "containerMap: removing container" podUID="793048dc-32db-41ad-9964-04c275d02cbf" containerName="istio-proxy"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330227    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="793048dc-32db-41ad-9964-04c275d02cbf" containerName="istio-proxy"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330240    5704 cpu_manager.go:414] "containerMap: removing container" podUID="353a50b4-be00-45b2-a6b1-af6498c0af69" containerName="payload-pre-processing"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330244    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="353a50b4-be00-45b2-a6b1-af6498c0af69" containerName="payload-pre-processing"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330330    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="d0289572-5d8d-4e2f-a14d-4bd58d050bd7" containerName="payload-processing"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330339    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="fda20ce9-bc4f-416d-8943-7fefd92f810a" containerName="istio-proxy"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330345    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="793048dc-32db-41ad-9964-04c275d02cbf" containerName="istio-proxy"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330351    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="a37355ab-b5c4-48af-b492-6536e09680dc" containerName="payload-pre-processing"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330360    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330366    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="46f7284e-cdd6-49c8-9f65-daff0f30f8bb" containerName="payload-pre-processing"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330373    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="793048dc-32db-41ad-9964-04c275d02cbf" containerName="istio-proxy"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330380    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="353a50b4-be00-45b2-a6b1-af6498c0af69" containerName="payload-pre-processing"
Oct 30 19:28:11.330405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.330388    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy"
Oct 30 19:28:11.335787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.335769    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"
Oct 30 19:28:11.408206 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.408178    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1466886-3b15-457a-a6c3-c17877af9aa1-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr\" (UID: \"a1466886-3b15-457a-a6c3-c17877af9aa1\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"
Oct 30 19:28:11.408379 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.408355    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5b6w\" (UniqueName: \"kubernetes.io/projected/a1466886-3b15-457a-a6c3-c17877af9aa1-kube-api-access-w5b6w\") pod \"payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr\" (UID: \"a1466886-3b15-457a-a6c3-c17877af9aa1\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"
Oct 30 19:28:11.467055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.456324    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-b-c16ba7\""
Oct 30 19:28:11.467597 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.467523    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-b-c16ba7-dockercfg-r5psr\""
Oct 30 19:28:11.478643 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.478573    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"]
Oct 30 19:28:11.485605 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.485569    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"]
Oct 30 19:28:11.486784 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.486755    5704 cpu_manager.go:414] "containerMap: removing container" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy"
Oct 30 19:28:11.486784 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.486784    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="096b94a8-ca2d-419e-9ce4-3554b762cbc8" containerName="istio-proxy"
Oct 30 19:28:11.491212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.491191    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"
Oct 30 19:28:11.509121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.509080    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1466886-3b15-457a-a6c3-c17877af9aa1-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr\" (UID: \"a1466886-3b15-457a-a6c3-c17877af9aa1\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"
Oct 30 19:28:11.509282 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.509261    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5b6w\" (UniqueName: \"kubernetes.io/projected/a1466886-3b15-457a-a6c3-c17877af9aa1-kube-api-access-w5b6w\") pod \"payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr\" (UID: \"a1466886-3b15-457a-a6c3-c17877af9aa1\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"
Oct 30 19:28:11.509760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.509742    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1466886-3b15-457a-a6c3-c17877af9aa1-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr\" (UID: \"a1466886-3b15-457a-a6c3-c17877af9aa1\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"
Oct 30 19:28:11.536221 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.536197    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"]
Oct 30 19:28:11.568282 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.568257    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5b6w\" (UniqueName: \"kubernetes.io/projected/a1466886-3b15-457a-a6c3-c17877af9aa1-kube-api-access-w5b6w\") pod \"payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr\" (UID: \"a1466886-3b15-457a-a6c3-c17877af9aa1\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"
Oct 30 19:28:11.609928 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.609864    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwpjd\" (UniqueName: \"kubernetes.io/projected/3e87b294-1fba-42e8-ae62-66e3f401cbbf-kube-api-access-xwpjd\") pod \"payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77\" (UID: \"3e87b294-1fba-42e8-ae62-66e3f401cbbf\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"
Oct 30 19:28:11.610104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.610083    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e87b294-1fba-42e8-ae62-66e3f401cbbf-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77\" (UID: \"3e87b294-1fba-42e8-ae62-66e3f401cbbf\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"
Oct 30 19:28:11.648696 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.648673    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"
Oct 30 19:28:11.712530 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.712439    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e87b294-1fba-42e8-ae62-66e3f401cbbf-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77\" (UID: \"3e87b294-1fba-42e8-ae62-66e3f401cbbf\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"
Oct 30 19:28:11.728525 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.721148    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwpjd\" (UniqueName: \"kubernetes.io/projected/3e87b294-1fba-42e8-ae62-66e3f401cbbf-kube-api-access-xwpjd\") pod \"payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77\" (UID: \"3e87b294-1fba-42e8-ae62-66e3f401cbbf\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"
Oct 30 19:28:11.728525 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.713853    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e87b294-1fba-42e8-ae62-66e3f401cbbf-plugins-config-volume\") pod \"payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77\" (UID: \"3e87b294-1fba-42e8-ae62-66e3f401cbbf\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"
Oct 30 19:28:11.779658 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.779628    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwpjd\" (UniqueName: \"kubernetes.io/projected/3e87b294-1fba-42e8-ae62-66e3f401cbbf-kube-api-access-xwpjd\") pod \"payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77\" (UID: \"3e87b294-1fba-42e8-ae62-66e3f401cbbf\") " pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"
Oct 30 19:28:11.808845 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:11.808820    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"
Oct 30 19:28:12.019751 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:12.019731    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"]
Oct 30 19:28:12.021983 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:28:12.021955    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1466886_3b15_457a_a6c3_c17877af9aa1.slice/crio-06cf3d72618741dd0dc980d274caec27c46ca11ce34890bd9ab6fe3cdb533e09 WatchSource:0}: Error finding container 06cf3d72618741dd0dc980d274caec27c46ca11ce34890bd9ab6fe3cdb533e09: Status 404 returned error can't find the container with id 06cf3d72618741dd0dc980d274caec27c46ca11ce34890bd9ab6fe3cdb533e09
Oct 30 19:28:12.177269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:12.177247    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"]
Oct 30 19:28:12.179279 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:28:12.179250    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e87b294_1fba_42e8_ae62_66e3f401cbbf.slice/crio-4266ee74a2b8234ce5767bef44a61848d67fb24ef2c48fb05459f13c566bc33d WatchSource:0}: Error finding container 4266ee74a2b8234ce5767bef44a61848d67fb24ef2c48fb05459f13c566bc33d: Status 404 returned error can't find the container with id 4266ee74a2b8234ce5767bef44a61848d67fb24ef2c48fb05459f13c566bc33d
Oct 30 19:28:12.869609 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:12.869570    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:12.893092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:12.893058    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr" event={"ID":"a1466886-3b15-457a-a6c3-c17877af9aa1","Type":"ContainerStarted","Data":"da28bd655ef38cfb9f7f65ff601841d0504f112a5171cfb5140f1d0e0de08077"}
Oct 30 19:28:12.893243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:12.893116    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr" event={"ID":"a1466886-3b15-457a-a6c3-c17877af9aa1","Type":"ContainerStarted","Data":"06cf3d72618741dd0dc980d274caec27c46ca11ce34890bd9ab6fe3cdb533e09"}
Oct 30 19:28:12.893243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:12.893156    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"
Oct 30 19:28:12.894756 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:12.894731    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77" event={"ID":"3e87b294-1fba-42e8-ae62-66e3f401cbbf","Type":"ContainerStarted","Data":"ed623f1c822348ef1b232b7cb8d309b6f0d79e3789e855a6d701609f38833c6c"}
Oct 30 19:28:12.894864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:12.894774    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"
Oct 30 19:28:12.894864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:12.894790    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77" event={"ID":"3e87b294-1fba-42e8-ae62-66e3f401cbbf","Type":"ContainerStarted","Data":"4266ee74a2b8234ce5767bef44a61848d67fb24ef2c48fb05459f13c566bc33d"}
Oct 30 19:28:13.017529 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.017479    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr" podStartSLOduration=2.017467288 podStartE2EDuration="2.017467288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:13.015273468 +0000 UTC m=+2465.961441364" watchObservedRunningTime="2026-10-30 19:28:13.017467288 +0000 UTC m=+2465.963635190"
Oct 30 19:28:13.125824 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.125721    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77" podStartSLOduration=2.125709234 podStartE2EDuration="2.125709234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:13.124158053 +0000 UTC m=+2466.070325957" watchObservedRunningTime="2026-10-30 19:28:13.125709234 +0000 UTC m=+2466.071877137"
Oct 30 19:28:13.228340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.228309    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"]
Oct 30 19:28:13.228510 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.228491    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r" podUID="f1d49546-da6c-445c-a491-3205a4daf0a8" containerName="payload-processing" containerID="cri-o://b9b66ff89396e73438b42c0c8e03194aaea6595764490af59184caee50330b2b" gracePeriod=30
Oct 30 19:28:13.472198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.472161    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:13.499502 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.499474    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"
Oct 30 19:28:13.655692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.655661    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1d49546-da6c-445c-a491-3205a4daf0a8-kube-api-access-v9kkb\" (UniqueName: \"kubernetes.io/projected/f1d49546-da6c-445c-a491-3205a4daf0a8-kube-api-access-v9kkb\") pod \"f1d49546-da6c-445c-a491-3205a4daf0a8\" (UID: \"f1d49546-da6c-445c-a491-3205a4daf0a8\") "
Oct 30 19:28:13.655859 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.655847    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1d49546-da6c-445c-a491-3205a4daf0a8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1d49546-da6c-445c-a491-3205a4daf0a8-plugins-config-volume\") pod \"f1d49546-da6c-445c-a491-3205a4daf0a8\" (UID: \"f1d49546-da6c-445c-a491-3205a4daf0a8\") "
Oct 30 19:28:13.656231 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.656207    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1d49546-da6c-445c-a491-3205a4daf0a8-plugins-config-volume" pod "f1d49546-da6c-445c-a491-3205a4daf0a8" (UID: "f1d49546-da6c-445c-a491-3205a4daf0a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:28:13.657860 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.657839    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1d49546-da6c-445c-a491-3205a4daf0a8-kube-api-access-v9kkb" pod "f1d49546-da6c-445c-a491-3205a4daf0a8" (UID: "f1d49546-da6c-445c-a491-3205a4daf0a8"). InnerVolumeSpecName "kube-api-access-v9kkb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:28:13.725054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.713766    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:13.757268 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.757232    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1d49546-da6c-445c-a491-3205a4daf0a8-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:13.757268 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.757267    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9kkb\" (UniqueName: \"kubernetes.io/projected/f1d49546-da6c-445c-a491-3205a4daf0a8-kube-api-access-v9kkb\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:13.902096 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.902059    5704 generic.go:356] "Generic (PLEG): container finished" podID="f1d49546-da6c-445c-a491-3205a4daf0a8" containerID="b9b66ff89396e73438b42c0c8e03194aaea6595764490af59184caee50330b2b" exitCode=0
Oct 30 19:28:13.902275 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.902100    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r" event={"ID":"f1d49546-da6c-445c-a491-3205a4daf0a8","Type":"ContainerDied","Data":"b9b66ff89396e73438b42c0c8e03194aaea6595764490af59184caee50330b2b"}
Oct 30 19:28:13.902275 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.902168    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r" event={"ID":"f1d49546-da6c-445c-a491-3205a4daf0a8","Type":"ContainerDied","Data":"72e055d0643478f1abd262dfd5ccd26dd800f58dcc23f81cd60855faa78fc4ae"}
Oct 30 19:28:13.902275 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.902117    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"
Oct 30 19:28:13.902401 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.902197    5704 scope.go:122] "RemoveContainer" containerID="b9b66ff89396e73438b42c0c8e03194aaea6595764490af59184caee50330b2b"
Oct 30 19:28:13.913695 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.913679    5704 scope.go:122] "RemoveContainer" containerID="b9b66ff89396e73438b42c0c8e03194aaea6595764490af59184caee50330b2b"
Oct 30 19:28:13.913953 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:28:13.913937    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9b66ff89396e73438b42c0c8e03194aaea6595764490af59184caee50330b2b\": container with ID starting with b9b66ff89396e73438b42c0c8e03194aaea6595764490af59184caee50330b2b not found: ID does not exist" containerID="b9b66ff89396e73438b42c0c8e03194aaea6595764490af59184caee50330b2b"
Oct 30 19:28:13.914006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:13.913962    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9b66ff89396e73438b42c0c8e03194aaea6595764490af59184caee50330b2b"} err="failed to get container status \"b9b66ff89396e73438b42c0c8e03194aaea6595764490af59184caee50330b2b\": rpc error: code = NotFound desc = could not find container \"b9b66ff89396e73438b42c0c8e03194aaea6595764490af59184caee50330b2b\": container with ID starting with b9b66ff89396e73438b42c0c8e03194aaea6595764490af59184caee50330b2b not found: ID does not exist"
Oct 30 19:28:14.057685 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:14.057655    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"]
Oct 30 19:28:14.060185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:14.060159    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-539084-59f8fcb766-bvq8r"]
Oct 30 19:28:14.539303 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:14.539264    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:14.787509 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:14.786610    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:15.008154 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:15.008121    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:15.180581 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:15.180551    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1d49546-da6c-445c-a491-3205a4daf0a8" path="/var/lib/kubelet/pods/f1d49546-da6c-445c-a491-3205a4daf0a8/volumes"
Oct 30 19:28:16.110237 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:16.110203    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:18.680364 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:18.680105    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:19.145595 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:19.145556    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:20.259400 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.259367    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"]
Oct 30 19:28:20.259953 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.259935    5704 cpu_manager.go:414] "containerMap: removing container" podUID="f1d49546-da6c-445c-a491-3205a4daf0a8" containerName="payload-processing"
Oct 30 19:28:20.260085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.259953    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1d49546-da6c-445c-a491-3205a4daf0a8" containerName="payload-processing"
Oct 30 19:28:20.260085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.260084    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1d49546-da6c-445c-a491-3205a4daf0a8" containerName="payload-processing"
Oct 30 19:28:20.264741 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.264723    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"
Oct 30 19:28:20.328318 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.328289    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwv9s\" (UniqueName: \"kubernetes.io/projected/d50a816f-dd92-4f8a-ae64-8b4ccce066de-kube-api-access-hwv9s\") pod \"payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd\" (UID: \"d50a816f-dd92-4f8a-ae64-8b4ccce066de\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"
Oct 30 19:28:20.328450 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.328422    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d50a816f-dd92-4f8a-ae64-8b4ccce066de-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd\" (UID: \"d50a816f-dd92-4f8a-ae64-8b4ccce066de\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"
Oct 30 19:28:20.360989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.360960    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-2a5c13-dockercfg-4474t\""
Oct 30 19:28:20.360989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.360956    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-2a5c13\""
Oct 30 19:28:20.363246 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.363223    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"]
Oct 30 19:28:20.430038 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.430013    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwv9s\" (UniqueName: \"kubernetes.io/projected/d50a816f-dd92-4f8a-ae64-8b4ccce066de-kube-api-access-hwv9s\") pod \"payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd\" (UID: \"d50a816f-dd92-4f8a-ae64-8b4ccce066de\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"
Oct 30 19:28:20.430178 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.430097    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d50a816f-dd92-4f8a-ae64-8b4ccce066de-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd\" (UID: \"d50a816f-dd92-4f8a-ae64-8b4ccce066de\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"
Oct 30 19:28:20.430762 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.430742    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d50a816f-dd92-4f8a-ae64-8b4ccce066de-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd\" (UID: \"d50a816f-dd92-4f8a-ae64-8b4ccce066de\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"
Oct 30 19:28:20.487861 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.487837    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwv9s\" (UniqueName: \"kubernetes.io/projected/d50a816f-dd92-4f8a-ae64-8b4ccce066de-kube-api-access-hwv9s\") pod \"payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd\" (UID: \"d50a816f-dd92-4f8a-ae64-8b4ccce066de\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"
Oct 30 19:28:20.577639 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.577576    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"
Oct 30 19:28:20.946148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:20.946124    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"]
Oct 30 19:28:20.948412 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:28:20.948383    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd50a816f_dd92_4f8a_ae64_8b4ccce066de.slice/crio-0616231d4822529de0a21419840fcb2aa72f82638e2bb86d4e2bc76b15c8965a WatchSource:0}: Error finding container 0616231d4822529de0a21419840fcb2aa72f82638e2bb86d4e2bc76b15c8965a: Status 404 returned error can't find the container with id 0616231d4822529de0a21419840fcb2aa72f82638e2bb86d4e2bc76b15c8965a
Oct 30 19:28:21.024293 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.024271    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"]
Oct 30 19:28:21.028856 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.028835    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"
Oct 30 19:28:21.087682 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.087655    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"]
Oct 30 19:28:21.139627 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.139602    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/516e0cbe-fd75-43ba-b635-c3427db43103-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz\" (UID: \"516e0cbe-fd75-43ba-b635-c3427db43103\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"
Oct 30 19:28:21.139777 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.139639    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d79w\" (UniqueName: \"kubernetes.io/projected/516e0cbe-fd75-43ba-b635-c3427db43103-kube-api-access-4d79w\") pod \"payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz\" (UID: \"516e0cbe-fd75-43ba-b635-c3427db43103\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"
Oct 30 19:28:21.240745 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.240665    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/516e0cbe-fd75-43ba-b635-c3427db43103-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz\" (UID: \"516e0cbe-fd75-43ba-b635-c3427db43103\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"
Oct 30 19:28:21.240745 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.240728    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d79w\" (UniqueName: \"kubernetes.io/projected/516e0cbe-fd75-43ba-b635-c3427db43103-kube-api-access-4d79w\") pod \"payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz\" (UID: \"516e0cbe-fd75-43ba-b635-c3427db43103\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"
Oct 30 19:28:21.241508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.241486    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/516e0cbe-fd75-43ba-b635-c3427db43103-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz\" (UID: \"516e0cbe-fd75-43ba-b635-c3427db43103\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"
Oct 30 19:28:21.298416 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.298391    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d79w\" (UniqueName: \"kubernetes.io/projected/516e0cbe-fd75-43ba-b635-c3427db43103-kube-api-access-4d79w\") pod \"payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz\" (UID: \"516e0cbe-fd75-43ba-b635-c3427db43103\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"
Oct 30 19:28:21.342312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.342288    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"
Oct 30 19:28:21.715005 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.714980    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"]
Oct 30 19:28:21.717571 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:28:21.717533    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod516e0cbe_fd75_43ba_b635_c3427db43103.slice/crio-6a1d78a06888326891a64321132a71501f012baf379b79591a282517e9c7427b WatchSource:0}: Error finding container 6a1d78a06888326891a64321132a71501f012baf379b79591a282517e9c7427b: Status 404 returned error can't find the container with id 6a1d78a06888326891a64321132a71501f012baf379b79591a282517e9c7427b
Oct 30 19:28:21.785020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.784985    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:21.953448 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.953399    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd" event={"ID":"d50a816f-dd92-4f8a-ae64-8b4ccce066de","Type":"ContainerStarted","Data":"80321943d0932e3201a90cacba5c0f7bc473d6ccb34e465964454e30faf281f0"}
Oct 30 19:28:21.953670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.953509    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"
Oct 30 19:28:21.953670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.953533    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd" event={"ID":"d50a816f-dd92-4f8a-ae64-8b4ccce066de","Type":"ContainerStarted","Data":"0616231d4822529de0a21419840fcb2aa72f82638e2bb86d4e2bc76b15c8965a"}
Oct 30 19:28:21.955147 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.955118    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz" event={"ID":"516e0cbe-fd75-43ba-b635-c3427db43103","Type":"ContainerStarted","Data":"82756fbdbc15dc7b313c9ce06866cd3107ba7af18406dea6097821454d43baea"}
Oct 30 19:28:21.955282 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.955166    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz" event={"ID":"516e0cbe-fd75-43ba-b635-c3427db43103","Type":"ContainerStarted","Data":"6a1d78a06888326891a64321132a71501f012baf379b79591a282517e9c7427b"}
Oct 30 19:28:21.955282 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:21.955221    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"
Oct 30 19:28:22.064330 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:22.064277    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd" podStartSLOduration=2.064258298 podStartE2EDuration="2.064258298s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:22.062388281 +0000 UTC m=+2475.008556186" watchObservedRunningTime="2026-10-30 19:28:22.064258298 +0000 UTC m=+2475.010426206"
Oct 30 19:28:22.191069 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:22.190583    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz" podStartSLOduration=2.190563352 podStartE2EDuration="2.190563352s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:22.189359035 +0000 UTC m=+2475.135526939" watchObservedRunningTime="2026-10-30 19:28:22.190563352 +0000 UTC m=+2475.136731256"
Oct 30 19:28:22.753497 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:22.753460    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:28:22.869283 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:22.869247    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:23.105855 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:23.105753    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:23.286000 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:23.285967    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:23.905764 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:23.905728    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"
Oct 30 19:28:23.906197 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:23.905940    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"
Oct 30 19:28:24.770730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:24.770702    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:25.767591 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:25.767555    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:26.009936 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.009898    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:26.216788 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.216700    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"]
Oct 30 19:28:26.216958 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.216927    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb" podUID="d9d79b71-6123-441f-9464-41e50708ce97" containerName="payload-processing" containerID="cri-o://9627685dfa1f6298f0945588583a52f53414044335e4285a75b687e52a567a83" gracePeriod=30
Oct 30 19:28:26.232917 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.232878    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:26.414062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.413959    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:26.475334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.475279    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"
Oct 30 19:28:26.615377 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.615346    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d9d79b71-6123-441f-9464-41e50708ce97-kube-api-access-r7m2g\" (UniqueName: \"kubernetes.io/projected/d9d79b71-6123-441f-9464-41e50708ce97-kube-api-access-r7m2g\") pod \"d9d79b71-6123-441f-9464-41e50708ce97\" (UID: \"d9d79b71-6123-441f-9464-41e50708ce97\") "
Oct 30 19:28:26.615541 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.615526    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d9d79b71-6123-441f-9464-41e50708ce97-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9d79b71-6123-441f-9464-41e50708ce97-plugins-config-volume\") pod \"d9d79b71-6123-441f-9464-41e50708ce97\" (UID: \"d9d79b71-6123-441f-9464-41e50708ce97\") "
Oct 30 19:28:26.615909 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.615883    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9d79b71-6123-441f-9464-41e50708ce97-plugins-config-volume" pod "d9d79b71-6123-441f-9464-41e50708ce97" (UID: "d9d79b71-6123-441f-9464-41e50708ce97"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:28:26.618095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.618068    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9d79b71-6123-441f-9464-41e50708ce97-kube-api-access-r7m2g" pod "d9d79b71-6123-441f-9464-41e50708ce97" (UID: "d9d79b71-6123-441f-9464-41e50708ce97"). InnerVolumeSpecName "kube-api-access-r7m2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:28:26.630437 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.630414    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:26.720920 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.720883    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7m2g\" (UniqueName: \"kubernetes.io/projected/d9d79b71-6123-441f-9464-41e50708ce97-kube-api-access-r7m2g\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:26.721180 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.721161    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d9d79b71-6123-441f-9464-41e50708ce97-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:26.986986 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.986954    5704 generic.go:356] "Generic (PLEG): container finished" podID="d9d79b71-6123-441f-9464-41e50708ce97" containerID="9627685dfa1f6298f0945588583a52f53414044335e4285a75b687e52a567a83" exitCode=0
Oct 30 19:28:26.987526 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.987008    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"
Oct 30 19:28:26.987526 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.987050    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb" event={"ID":"d9d79b71-6123-441f-9464-41e50708ce97","Type":"ContainerDied","Data":"9627685dfa1f6298f0945588583a52f53414044335e4285a75b687e52a567a83"}
Oct 30 19:28:26.987526 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.987142    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb" event={"ID":"d9d79b71-6123-441f-9464-41e50708ce97","Type":"ContainerDied","Data":"bea51d41fc1771083c6f7559aef9a0f27d42d08763cb1d26936af6a4e445b0ff"}
Oct 30 19:28:26.987526 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.987175    5704 scope.go:122] "RemoveContainer" containerID="9627685dfa1f6298f0945588583a52f53414044335e4285a75b687e52a567a83"
Oct 30 19:28:26.997606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.997585    5704 scope.go:122] "RemoveContainer" containerID="9627685dfa1f6298f0945588583a52f53414044335e4285a75b687e52a567a83"
Oct 30 19:28:26.997872 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:28:26.997852    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9627685dfa1f6298f0945588583a52f53414044335e4285a75b687e52a567a83\": container with ID starting with 9627685dfa1f6298f0945588583a52f53414044335e4285a75b687e52a567a83 not found: ID does not exist" containerID="9627685dfa1f6298f0945588583a52f53414044335e4285a75b687e52a567a83"
Oct 30 19:28:26.997946 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:26.997879    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9627685dfa1f6298f0945588583a52f53414044335e4285a75b687e52a567a83"} err="failed to get container status \"9627685dfa1f6298f0945588583a52f53414044335e4285a75b687e52a567a83\": rpc error: code = NotFound desc = could not find container \"9627685dfa1f6298f0945588583a52f53414044335e4285a75b687e52a567a83\": container with ID starting with 9627685dfa1f6298f0945588583a52f53414044335e4285a75b687e52a567a83 not found: ID does not exist"
Oct 30 19:28:27.143433 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:27.143407    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"]
Oct 30 19:28:27.188294 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:27.188268    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-496c5b-5779878db6-69qlb"]
Oct 30 19:28:27.485366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:27.485325    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:27.689563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:27.689530    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:27.882113 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:27.882079    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:28.507862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:28.507830    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:29.179909 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:29.179880    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d9d79b71-6123-441f-9464-41e50708ce97" path="/var/lib/kubelet/pods/d9d79b71-6123-441f-9464-41e50708ce97/volumes"
Oct 30 19:28:30.619462 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:30.619433    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:30.929551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:30.929473    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:31.134073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:31.134014    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:32.963910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:32.963878    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"
Oct 30 19:28:32.964416 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:32.964298    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"
Oct 30 19:28:33.008624 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:33.008592    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:33.383857 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:28:33.383822    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:28:34.017259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:34.017230    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:34.230462 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:34.230432    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:36.584379 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:36.584346    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:36.775686 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:36.775658    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:37.022923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:37.022891    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:38.615413 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:38.615377    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:41.536469 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:41.536429    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:42.230269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:42.230229    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:42.426391 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:42.426354    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:44.408198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:44.408099    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:44.517251 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:44.517222    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"]
Oct 30 19:28:44.517431 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:44.517410    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v" podUID="a7704518-d998-47b2-8044-f87a597ce053" containerName="maas-api" containerID="cri-o://d5636554ad3b042af9deae1d3f7989775e97a56ae52ed069a5b880d522e95879" gracePeriod=30
Oct 30 19:28:44.668464 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:44.668388    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:44.808259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:44.808236    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:28:44.922669 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:44.922578    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-metrics-certs\") pod \"a7704518-d998-47b2-8044-f87a597ce053\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") "
Oct 30 19:28:44.922669 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:44.922639    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a7704518-d998-47b2-8044-f87a597ce053-kube-api-access-x9gvq\" (UniqueName: \"kubernetes.io/projected/a7704518-d998-47b2-8044-f87a597ce053-kube-api-access-x9gvq\") pod \"a7704518-d998-47b2-8044-f87a597ce053\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") "
Oct 30 19:28:44.922922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:44.922736    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-maas-api-tls\") pod \"a7704518-d998-47b2-8044-f87a597ce053\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") "
Oct 30 19:28:44.922922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:44.922860    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a7704518-d998-47b2-8044-f87a597ce053-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7704518-d998-47b2-8044-f87a597ce053-openshift-service-ca\") pod \"a7704518-d998-47b2-8044-f87a597ce053\" (UID: \"a7704518-d998-47b2-8044-f87a597ce053\") "
Oct 30 19:28:44.923531 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:44.923502    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7704518-d998-47b2-8044-f87a597ce053-openshift-service-ca" pod "a7704518-d998-47b2-8044-f87a597ce053" (UID: "a7704518-d998-47b2-8044-f87a597ce053"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:28:44.925297 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:44.925267    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-maas-api-tls" pod "a7704518-d998-47b2-8044-f87a597ce053" (UID: "a7704518-d998-47b2-8044-f87a597ce053"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:28:44.925414 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:44.925337    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7704518-d998-47b2-8044-f87a597ce053-kube-api-access-x9gvq" pod "a7704518-d998-47b2-8044-f87a597ce053" (UID: "a7704518-d998-47b2-8044-f87a597ce053"). InnerVolumeSpecName "kube-api-access-x9gvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:28:44.925481 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:44.925418    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-metrics-certs" pod "a7704518-d998-47b2-8044-f87a597ce053" (UID: "a7704518-d998-47b2-8044-f87a597ce053"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:28:45.024176 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:45.024144    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:45.024176 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:45.024174    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a7704518-d998-47b2-8044-f87a597ce053-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:45.024347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:45.024185    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7704518-d998-47b2-8044-f87a597ce053-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:45.024347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:45.024195    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9gvq\" (UniqueName: \"kubernetes.io/projected/a7704518-d998-47b2-8044-f87a597ce053-kube-api-access-x9gvq\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:28:45.095073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:45.095022    5704 generic.go:356] "Generic (PLEG): container finished" podID="a7704518-d998-47b2-8044-f87a597ce053" containerID="d5636554ad3b042af9deae1d3f7989775e97a56ae52ed069a5b880d522e95879" exitCode=0
Oct 30 19:28:45.095277 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:45.095091    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"
Oct 30 19:28:45.095277 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:45.095093    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v" event={"ID":"a7704518-d998-47b2-8044-f87a597ce053","Type":"ContainerDied","Data":"d5636554ad3b042af9deae1d3f7989775e97a56ae52ed069a5b880d522e95879"}
Oct 30 19:28:45.095277 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:45.095159    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v" event={"ID":"a7704518-d998-47b2-8044-f87a597ce053","Type":"ContainerDied","Data":"753b608d4bbe96b8425c4b6cd9bb56e66b28617976c19955725a605a996b4241"}
Oct 30 19:28:45.095277 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:45.095188    5704 scope.go:122] "RemoveContainer" containerID="d5636554ad3b042af9deae1d3f7989775e97a56ae52ed069a5b880d522e95879"
Oct 30 19:28:45.105267 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:45.105248    5704 scope.go:122] "RemoveContainer" containerID="d5636554ad3b042af9deae1d3f7989775e97a56ae52ed069a5b880d522e95879"
Oct 30 19:28:45.105587 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:28:45.105566    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5636554ad3b042af9deae1d3f7989775e97a56ae52ed069a5b880d522e95879\": container with ID starting with d5636554ad3b042af9deae1d3f7989775e97a56ae52ed069a5b880d522e95879 not found: ID does not exist" containerID="d5636554ad3b042af9deae1d3f7989775e97a56ae52ed069a5b880d522e95879"
Oct 30 19:28:45.105651 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:45.105595    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5636554ad3b042af9deae1d3f7989775e97a56ae52ed069a5b880d522e95879"} err="failed to get container status \"d5636554ad3b042af9deae1d3f7989775e97a56ae52ed069a5b880d522e95879\": rpc error: code = NotFound desc = could not find container \"d5636554ad3b042af9deae1d3f7989775e97a56ae52ed069a5b880d522e95879\": container with ID starting with d5636554ad3b042af9deae1d3f7989775e97a56ae52ed069a5b880d522e95879 not found: ID does not exist"
Oct 30 19:28:45.266792 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:45.266767    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"]
Oct 30 19:28:45.268808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:45.268784    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-d6b5ac-5c9b496bb-ddh6v"]
Oct 30 19:28:46.300019 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:46.299990    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:46.527521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:46.527484    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:46.744329 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:46.744239    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:47.179621 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:47.179590    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a7704518-d998-47b2-8044-f87a597ce053" path="/var/lib/kubelet/pods/a7704518-d998-47b2-8044-f87a597ce053/volumes"
Oct 30 19:28:47.353520 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:47.353487    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:47.613897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:47.613862    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:50.542729 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:50.542692    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:50.761858 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:50.761829    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:50.941163 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:50.941085    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"]
Oct 30 19:28:50.941646 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:50.941633    5704 cpu_manager.go:414] "containerMap: removing container" podUID="d9d79b71-6123-441f-9464-41e50708ce97" containerName="payload-processing"
Oct 30 19:28:50.941689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:50.941647    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d9d79b71-6123-441f-9464-41e50708ce97" containerName="payload-processing"
Oct 30 19:28:50.941689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:50.941673    5704 cpu_manager.go:414] "containerMap: removing container" podUID="a7704518-d998-47b2-8044-f87a597ce053" containerName="maas-api"
Oct 30 19:28:50.941689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:50.941678    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a7704518-d998-47b2-8044-f87a597ce053" containerName="maas-api"
Oct 30 19:28:50.941792 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:50.941769    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="d9d79b71-6123-441f-9464-41e50708ce97" containerName="payload-processing"
Oct 30 19:28:50.941792 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:50.941782    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="a7704518-d998-47b2-8044-f87a597ce053" containerName="maas-api"
Oct 30 19:28:50.946688 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:50.946667    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"
Oct 30 19:28:51.052073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.045911    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-441836\""
Oct 30 19:28:51.052073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.045911    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-441836-dockercfg-zvpbw\""
Oct 30 19:28:51.052073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.049776    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"]
Oct 30 19:28:51.094358 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.094327    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"]
Oct 30 19:28:51.094982 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.094953    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/020fbb14-fd8e-4941-9c3f-91309c71ce47-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj\" (UID: \"020fbb14-fd8e-4941-9c3f-91309c71ce47\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"
Oct 30 19:28:51.095154 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.095131    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsrrd\" (UniqueName: \"kubernetes.io/projected/020fbb14-fd8e-4941-9c3f-91309c71ce47-kube-api-access-xsrrd\") pod \"payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj\" (UID: \"020fbb14-fd8e-4941-9c3f-91309c71ce47\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"
Oct 30 19:28:51.098851 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.098833    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"
Oct 30 19:28:51.160486 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.160460    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"]
Oct 30 19:28:51.196159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.196083    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/020fbb14-fd8e-4941-9c3f-91309c71ce47-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj\" (UID: \"020fbb14-fd8e-4941-9c3f-91309c71ce47\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"
Oct 30 19:28:51.196317 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.196232    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsrrd\" (UniqueName: \"kubernetes.io/projected/020fbb14-fd8e-4941-9c3f-91309c71ce47-kube-api-access-xsrrd\") pod \"payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj\" (UID: \"020fbb14-fd8e-4941-9c3f-91309c71ce47\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"
Oct 30 19:28:51.196317 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.196287    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrkdv\" (UniqueName: \"kubernetes.io/projected/82681f31-bc24-4592-8f1f-5ea5b852e40a-kube-api-access-lrkdv\") pod \"payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292\" (UID: \"82681f31-bc24-4592-8f1f-5ea5b852e40a\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"
Oct 30 19:28:51.196434 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.196326    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82681f31-bc24-4592-8f1f-5ea5b852e40a-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292\" (UID: \"82681f31-bc24-4592-8f1f-5ea5b852e40a\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"
Oct 30 19:28:51.196828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.196810    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/020fbb14-fd8e-4941-9c3f-91309c71ce47-plugins-config-volume\") pod \"payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj\" (UID: \"020fbb14-fd8e-4941-9c3f-91309c71ce47\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"
Oct 30 19:28:51.258027 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.257992    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsrrd\" (UniqueName: \"kubernetes.io/projected/020fbb14-fd8e-4941-9c3f-91309c71ce47-kube-api-access-xsrrd\") pod \"payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj\" (UID: \"020fbb14-fd8e-4941-9c3f-91309c71ce47\") " pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"
Oct 30 19:28:51.260404 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.260388    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"
Oct 30 19:28:51.298148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.298115    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrkdv\" (UniqueName: \"kubernetes.io/projected/82681f31-bc24-4592-8f1f-5ea5b852e40a-kube-api-access-lrkdv\") pod \"payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292\" (UID: \"82681f31-bc24-4592-8f1f-5ea5b852e40a\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"
Oct 30 19:28:51.298301 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.298154    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82681f31-bc24-4592-8f1f-5ea5b852e40a-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292\" (UID: \"82681f31-bc24-4592-8f1f-5ea5b852e40a\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"
Oct 30 19:28:51.298826 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.298806    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82681f31-bc24-4592-8f1f-5ea5b852e40a-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292\" (UID: \"82681f31-bc24-4592-8f1f-5ea5b852e40a\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"
Oct 30 19:28:51.381055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.380636    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrkdv\" (UniqueName: \"kubernetes.io/projected/82681f31-bc24-4592-8f1f-5ea5b852e40a-kube-api-access-lrkdv\") pod \"payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292\" (UID: \"82681f31-bc24-4592-8f1f-5ea5b852e40a\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"
Oct 30 19:28:51.413022 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.412982    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"
Oct 30 19:28:51.635535 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.635504    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"]
Oct 30 19:28:51.640194 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.640176    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:28:51.642972 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:28:51.642947    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod020fbb14_fd8e_4941_9c3f_91309c71ce47.slice/crio-16441d7f158b57d0f95ba3e05ff6c773cdb26566e057c3d260205a05476d96e1 WatchSource:0}: Error finding container 16441d7f158b57d0f95ba3e05ff6c773cdb26566e057c3d260205a05476d96e1: Status 404 returned error can't find the container with id 16441d7f158b57d0f95ba3e05ff6c773cdb26566e057c3d260205a05476d96e1
Oct 30 19:28:51.808947 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.808919    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"]
Oct 30 19:28:51.815124 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:28:51.815084    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82681f31_bc24_4592_8f1f_5ea5b852e40a.slice/crio-d5ea9e8649b597f476af8b9aa7b1fb5b4de9695224615a274139533f7cc4f547 WatchSource:0}: Error finding container d5ea9e8649b597f476af8b9aa7b1fb5b4de9695224615a274139533f7cc4f547: Status 404 returned error can't find the container with id d5ea9e8649b597f476af8b9aa7b1fb5b4de9695224615a274139533f7cc4f547
Oct 30 19:28:51.934662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:51.934632    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:52.157946 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:52.157835    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292" event={"ID":"82681f31-bc24-4592-8f1f-5ea5b852e40a","Type":"ContainerStarted","Data":"36f3c4a69e89a655f8e14238836540e1d839c80f520c3b6592361b69f07bc1cf"}
Oct 30 19:28:52.157946 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:52.157907    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292" event={"ID":"82681f31-bc24-4592-8f1f-5ea5b852e40a","Type":"ContainerStarted","Data":"d5ea9e8649b597f476af8b9aa7b1fb5b4de9695224615a274139533f7cc4f547"}
Oct 30 19:28:52.159177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:52.159155    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"
Oct 30 19:28:52.160797 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:52.160179    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj" event={"ID":"020fbb14-fd8e-4941-9c3f-91309c71ce47","Type":"ContainerStarted","Data":"c805cd3852f9317f23e53c0454d4e8d71bc469e7f2667bd7e65ba6aa64adcb09"}
Oct 30 19:28:52.160797 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:52.160228    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj" event={"ID":"020fbb14-fd8e-4941-9c3f-91309c71ce47","Type":"ContainerStarted","Data":"16441d7f158b57d0f95ba3e05ff6c773cdb26566e057c3d260205a05476d96e1"}
Oct 30 19:28:52.160797 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:52.160256    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"
Oct 30 19:28:52.276754 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:52.276708    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292" podStartSLOduration=1.2766955260000001 podStartE2EDuration="1.276695526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:52.275537306 +0000 UTC m=+2505.221705221" watchObservedRunningTime="2026-10-30 19:28:52.276695526 +0000 UTC m=+2505.222863429"
Oct 30 19:28:52.397941 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:52.397894    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj" podStartSLOduration=2.39788111 podStartE2EDuration="2.39788111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:28:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:28:52.395837218 +0000 UTC m=+2505.342005126" watchObservedRunningTime="2026-10-30 19:28:52.39788111 +0000 UTC m=+2505.344049013"
Oct 30 19:28:54.752219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:54.752178    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:55.059703 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:55.059625    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:55.513427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:55.513383    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:56.587692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:56.587647    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:56.844660 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:56.844572    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:57.274311 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:57.274272    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:28:57.455853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:28:57.455816    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:01.659812 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:01.656748    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:01.852383 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:01.852341    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:02.030215 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:02.030178    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:03.169682 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:03.169653    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"
Oct 30 19:29:03.383868 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:29:03.383833    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:29:04.174681 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:04.174654    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"
Oct 30 19:29:05.161311 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:05.161270    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:06.360132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:06.360098    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:06.652641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:06.652554    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:08.508324 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:08.508290    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:11.295221 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:11.295187    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:11.748269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:11.748236    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:11.989259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:11.989225    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:14.881885 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:14.881846    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:15.087642 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:15.087610    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:18.117721 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:18.117606    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:21.056891 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:21.056845    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:21.295580 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:21.295538    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:23.066308 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:23.066273    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:27.970331 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:27.970297    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:28.160341 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:28.160304    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:28.362152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:28.362117    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:30.589545 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:30.589510    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:33.088923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:33.088880    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:33.384351 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:29:33.384271    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:29:37.916273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:37.916238    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:38.567650 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:38.567617    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:38.756742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:38.756707    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:43.307969 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.307938    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:29:43.795312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.795285    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"]
Oct 30 19:29:43.799592 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.799573    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:43.848942 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.848918    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-a-441836-serving-cert\""
Oct 30 19:29:43.892157 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.892132    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz66v\" (UniqueName: \"kubernetes.io/projected/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-kube-api-access-mz66v\") pod \"maas-api-e2e-praxis-a-441836-865c866495-czznc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:43.892322 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.892202    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-441836-865c866495-czznc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:43.892394 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.892340    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-maas-api-tls\") pod \"maas-api-e2e-praxis-a-441836-865c866495-czznc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:43.892457 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.892441    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-metrics-certs\") pod \"maas-api-e2e-praxis-a-441836-865c866495-czznc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:43.897284 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.897263    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"]
Oct 30 19:29:43.993189 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.993161    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz66v\" (UniqueName: \"kubernetes.io/projected/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-kube-api-access-mz66v\") pod \"maas-api-e2e-praxis-a-441836-865c866495-czznc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:43.993362 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.993319    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-441836-865c866495-czznc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:43.993438 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.993376    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-maas-api-tls\") pod \"maas-api-e2e-praxis-a-441836-865c866495-czznc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:43.993438 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.993425    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-metrics-certs\") pod \"maas-api-e2e-praxis-a-441836-865c866495-czznc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:43.993542 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:29:43.993513    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-serving-cert: secret "maas-api-e2e-praxis-a-441836-serving-cert" not found
Oct 30 19:29:43.993609 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:29:43.993598    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-maas-api-tls podName:e44e6675-761e-427f-b3b5-4ccbd2bb0dbc nodeName:}" failed. No retries permitted until 2026-10-30 19:29:44.493581131 +0000 UTC m=+2557.439749025 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-maas-api-tls") pod "maas-api-e2e-praxis-a-441836-865c866495-czznc" (UID: "e44e6675-761e-427f-b3b5-4ccbd2bb0dbc") : secret "maas-api-e2e-praxis-a-441836-serving-cert" not found
Oct 30 19:29:43.993929 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.993912    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-openshift-service-ca\") pod \"maas-api-e2e-praxis-a-441836-865c866495-czznc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:43.995817 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:43.995795    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-metrics-certs\") pod \"maas-api-e2e-praxis-a-441836-865c866495-czznc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:44.024371 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:44.024343    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:44.052508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:44.052454    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz66v\" (UniqueName: \"kubernetes.io/projected/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-kube-api-access-mz66v\") pod \"maas-api-e2e-praxis-a-441836-865c866495-czznc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:44.277320 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:44.277285    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:44.499725 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:44.499693    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-maas-api-tls\") pod \"maas-api-e2e-praxis-a-441836-865c866495-czznc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:44.502166 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:44.502142    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-maas-api-tls\") pod \"maas-api-e2e-praxis-a-441836-865c866495-czznc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:44.712431 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:44.712400    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:45.090475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:45.090399    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"]
Oct 30 19:29:45.097148 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:29:45.097114    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode44e6675_761e_427f_b3b5_4ccbd2bb0dbc.slice/crio-d45c4f68fcacd245073acd604a9d3442a34e0854212a85f20f639decc76c728f WatchSource:0}: Error finding container d45c4f68fcacd245073acd604a9d3442a34e0854212a85f20f639decc76c728f: Status 404 returned error can't find the container with id d45c4f68fcacd245073acd604a9d3442a34e0854212a85f20f639decc76c728f
Oct 30 19:29:45.484538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:45.484451    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc" event={"ID":"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc","Type":"ContainerStarted","Data":"d45c4f68fcacd245073acd604a9d3442a34e0854212a85f20f639decc76c728f"}
Oct 30 19:29:46.492803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:46.492771    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc" event={"ID":"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc","Type":"ContainerStarted","Data":"161040bcd6004fff71dfd0ed14f9679328d80fbc65bf3daa5cfd9459b75b0972"}
Oct 30 19:29:46.493259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:46.492859    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:46.605242 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:46.605192    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc" podStartSLOduration=2.824740216 podStartE2EDuration="3.60517927s" totalImagesPullingTime="780.439054ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:29:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:29:46.60386382 +0000 UTC m=+2559.550031727" watchObservedRunningTime="2026-10-30 19:29:46.60517927 +0000 UTC m=+2559.551347173"
Oct 30 19:29:48.934622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:48.934584    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:52.503868 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:52.503842    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:29:53.569632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:53.569598    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:29:58.274679 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:29:58.274642    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:02.970608 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:02.970568    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:03.383947 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:30:03.383915    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:30:07.722707 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:07.722652    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:08.713209 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:08.713177    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:30:08.946933 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:08.946902    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:11.470162 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:11.470127    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:12.253105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.253073    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"]
Oct 30 19:30:12.257260 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.257239    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:12.356289 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.356263    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"]
Oct 30 19:30:12.361245 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.361224    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-f3b5af-serving-cert\""
Oct 30 19:30:12.398881 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.398843    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-metrics-certs\") pod \"maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:12.399070 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.398897    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:12.399070 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.398977    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c84578b-0f06-46c9-a103-9f01bd06e5e7-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:12.399282 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.399260    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krh5z\" (UniqueName: \"kubernetes.io/projected/8c84578b-0f06-46c9-a103-9f01bd06e5e7-kube-api-access-krh5z\") pod \"maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:12.464578 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.464551    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:12.500124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.500083    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krh5z\" (UniqueName: \"kubernetes.io/projected/8c84578b-0f06-46c9-a103-9f01bd06e5e7-kube-api-access-krh5z\") pod \"maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:12.500604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.500161    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-metrics-certs\") pod \"maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:12.500604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.500188    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:12.500604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.500202    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c84578b-0f06-46c9-a103-9f01bd06e5e7-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:12.500781 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:30:12.500614    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-serving-cert: secret "maas-api-e2e-ipp-a-f3b5af-serving-cert" not found
Oct 30 19:30:12.500781 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:30:12.500662    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-maas-api-tls podName:8c84578b-0f06-46c9-a103-9f01bd06e5e7 nodeName:}" failed. No retries permitted until 2026-10-30 19:30:13.000648592 +0000 UTC m=+2585.946816490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-maas-api-tls") pod "maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8" (UID: "8c84578b-0f06-46c9-a103-9f01bd06e5e7") : secret "maas-api-e2e-ipp-a-f3b5af-serving-cert" not found
Oct 30 19:30:12.500917 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.500833    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c84578b-0f06-46c9-a103-9f01bd06e5e7-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:12.503726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.503656    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-metrics-certs\") pod \"maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:12.558385 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.558356    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-krh5z\" (UniqueName: \"kubernetes.io/projected/8c84578b-0f06-46c9-a103-9f01bd06e5e7-kube-api-access-krh5z\") pod \"maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:12.686179 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.686149    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:12.868662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:12.868589    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:13.005374 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:13.005342    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:13.007663 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:13.007636    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-maas-api-tls\") pod \"maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:13.170511 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:13.170447    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:13.547722 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:13.547691    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"]
Oct 30 19:30:13.554676 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:30:13.554649    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c84578b_0f06_46c9_a103_9f01bd06e5e7.slice/crio-80ee0356a09351dca9f0e16e6ec7f3ba9d4db3c7672e95675d344fc2128fedb5 WatchSource:0}: Error finding container 80ee0356a09351dca9f0e16e6ec7f3ba9d4db3c7672e95675d344fc2128fedb5: Status 404 returned error can't find the container with id 80ee0356a09351dca9f0e16e6ec7f3ba9d4db3c7672e95675d344fc2128fedb5
Oct 30 19:30:13.657943 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:13.657910    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8" event={"ID":"8c84578b-0f06-46c9-a103-9f01bd06e5e7","Type":"ContainerStarted","Data":"80ee0356a09351dca9f0e16e6ec7f3ba9d4db3c7672e95675d344fc2128fedb5"}
Oct 30 19:30:14.666596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:14.666515    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8" event={"ID":"8c84578b-0f06-46c9-a103-9f01bd06e5e7","Type":"ContainerStarted","Data":"39d62b82fe92981b4cf4a460e37bd3f93bc82a11f71dcaebb3afcbdd146c6e62"}
Oct 30 19:30:14.666975 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:14.666639    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:14.778720 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:14.778673    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8" podStartSLOduration=1.959580035 podStartE2EDuration="2.778657841s" totalImagesPullingTime="819.077806ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:30:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:30:14.778260707 +0000 UTC m=+2587.724428613" watchObservedRunningTime="2026-10-30 19:30:14.778657841 +0000 UTC m=+2587.724825742"
Oct 30 19:30:17.509256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:17.509215    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:20.677565 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:20.677537    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:30:23.917777 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:23.917734    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:28.729431 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:28.729394    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:33.382973 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:30:33.382937    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:30:33.896973 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:33.896939    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:37.860432 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:37.860396    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"]
Oct 30 19:30:38.132562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:38.132476    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:38.305915 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:38.305879    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:38.829396 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:38.829359    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:42.861392 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:42.861353    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l" podUID="702caeab-5ddb-4a78-b1a2-80993766a989" containerName="payload-pre-processing" containerID="cri-o://5128ef619b166cd20ac37ea0a421ad542672157461cfaa13032d2a46063d1570" gracePeriod=25
Oct 30 19:30:43.118678 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.118614    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"
Oct 30 19:30:43.269433 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.269395    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/702caeab-5ddb-4a78-b1a2-80993766a989-kube-api-access-zs9l4\" (UniqueName: \"kubernetes.io/projected/702caeab-5ddb-4a78-b1a2-80993766a989-kube-api-access-zs9l4\") pod \"702caeab-5ddb-4a78-b1a2-80993766a989\" (UID: \"702caeab-5ddb-4a78-b1a2-80993766a989\") "
Oct 30 19:30:43.269616 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.269442    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/702caeab-5ddb-4a78-b1a2-80993766a989-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/702caeab-5ddb-4a78-b1a2-80993766a989-plugins-config-volume\") pod \"702caeab-5ddb-4a78-b1a2-80993766a989\" (UID: \"702caeab-5ddb-4a78-b1a2-80993766a989\") "
Oct 30 19:30:43.269886 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.269856    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/702caeab-5ddb-4a78-b1a2-80993766a989-plugins-config-volume" pod "702caeab-5ddb-4a78-b1a2-80993766a989" (UID: "702caeab-5ddb-4a78-b1a2-80993766a989"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:43.271566 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.271547    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/702caeab-5ddb-4a78-b1a2-80993766a989-kube-api-access-zs9l4" pod "702caeab-5ddb-4a78-b1a2-80993766a989" (UID: "702caeab-5ddb-4a78-b1a2-80993766a989"). InnerVolumeSpecName "kube-api-access-zs9l4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:43.370654 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.370586    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs9l4\" (UniqueName: \"kubernetes.io/projected/702caeab-5ddb-4a78-b1a2-80993766a989-kube-api-access-zs9l4\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:43.370654 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.370609    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/702caeab-5ddb-4a78-b1a2-80993766a989-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:43.841895 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.841860    5704 generic.go:356] "Generic (PLEG): container finished" podID="702caeab-5ddb-4a78-b1a2-80993766a989" containerID="5128ef619b166cd20ac37ea0a421ad542672157461cfaa13032d2a46063d1570" exitCode=0
Oct 30 19:30:43.842177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.841902    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"
Oct 30 19:30:43.842177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.841946    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l" event={"ID":"702caeab-5ddb-4a78-b1a2-80993766a989","Type":"ContainerDied","Data":"5128ef619b166cd20ac37ea0a421ad542672157461cfaa13032d2a46063d1570"}
Oct 30 19:30:43.842177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.842008    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l" event={"ID":"702caeab-5ddb-4a78-b1a2-80993766a989","Type":"ContainerDied","Data":"a90c99ea5a3ca43ac97fd135b6f3f23e0fb7b51b887de2d70e24cef2d5c3a44b"}
Oct 30 19:30:43.842177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.842055    5704 scope.go:122] "RemoveContainer" containerID="5128ef619b166cd20ac37ea0a421ad542672157461cfaa13032d2a46063d1570"
Oct 30 19:30:43.852318 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.852302    5704 scope.go:122] "RemoveContainer" containerID="5128ef619b166cd20ac37ea0a421ad542672157461cfaa13032d2a46063d1570"
Oct 30 19:30:43.852596 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:30:43.852569    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5128ef619b166cd20ac37ea0a421ad542672157461cfaa13032d2a46063d1570\": container with ID starting with 5128ef619b166cd20ac37ea0a421ad542672157461cfaa13032d2a46063d1570 not found: ID does not exist" containerID="5128ef619b166cd20ac37ea0a421ad542672157461cfaa13032d2a46063d1570"
Oct 30 19:30:43.852655 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.852604    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5128ef619b166cd20ac37ea0a421ad542672157461cfaa13032d2a46063d1570"} err="failed to get container status \"5128ef619b166cd20ac37ea0a421ad542672157461cfaa13032d2a46063d1570\": rpc error: code = NotFound desc = could not find container \"5128ef619b166cd20ac37ea0a421ad542672157461cfaa13032d2a46063d1570\": container with ID starting with 5128ef619b166cd20ac37ea0a421ad542672157461cfaa13032d2a46063d1570 not found: ID does not exist"
Oct 30 19:30:43.974828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.974799    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:43.999215 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:43.999182    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"]
Oct 30 19:30:44.045298 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:44.045269    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-4833f3-8485b7b49f-zhr5l"]
Oct 30 19:30:44.321619 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:44.321590    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:45.180242 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:45.180208    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="702caeab-5ddb-4a78-b1a2-80993766a989" path="/var/lib/kubelet/pods/702caeab-5ddb-4a78-b1a2-80993766a989/volumes"
Oct 30 19:30:45.187574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:45.187546    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:45.863340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:45.863305    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:46.038532 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:46.038497    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:50.753203 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:50.753119    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:50.985146 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:50.985109    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:51.183522 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:51.183439    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:53.904617 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:53.904582    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:54.082874 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:54.082839    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:55.221950 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.221913    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"]
Oct 30 19:30:55.222482 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.222179    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw" podUID="e1de5a60-9b29-46a8-a318-a3ff1eb48b32" containerName="maas-api" containerID="cri-o://a9909badd1b3bcb363a4f9af6719b006571c48fd13e62f72edef1a73ab73c2eb" gracePeriod=30
Oct 30 19:30:55.378752 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.378718    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:55.501786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.501759    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:30:55.619937 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.619909    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-openshift-service-ca\") pod \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") "
Oct 30 19:30:55.620129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.619986    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-maas-api-tls\") pod \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") "
Oct 30 19:30:55.620216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.620130    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-kube-api-access-x5g2z\" (UniqueName: \"kubernetes.io/projected/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-kube-api-access-x5g2z\") pod \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") "
Oct 30 19:30:55.620216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.620167    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-metrics-certs\") pod \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\" (UID: \"e1de5a60-9b29-46a8-a318-a3ff1eb48b32\") "
Oct 30 19:30:55.620396 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.620359    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-openshift-service-ca" pod "e1de5a60-9b29-46a8-a318-a3ff1eb48b32" (UID: "e1de5a60-9b29-46a8-a318-a3ff1eb48b32"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:30:55.620673 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.620620    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:55.623402 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.623362    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-maas-api-tls" pod "e1de5a60-9b29-46a8-a318-a3ff1eb48b32" (UID: "e1de5a60-9b29-46a8-a318-a3ff1eb48b32"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:30:55.645099 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.635861    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-kube-api-access-x5g2z" pod "e1de5a60-9b29-46a8-a318-a3ff1eb48b32" (UID: "e1de5a60-9b29-46a8-a318-a3ff1eb48b32"). InnerVolumeSpecName "kube-api-access-x5g2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:30:55.645099 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.636219    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-metrics-certs" pod "e1de5a60-9b29-46a8-a318-a3ff1eb48b32" (UID: "e1de5a60-9b29-46a8-a318-a3ff1eb48b32"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:30:55.722355 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.722324    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:55.722355 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.722350    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5g2z\" (UniqueName: \"kubernetes.io/projected/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-kube-api-access-x5g2z\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:55.722355 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.722361    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1de5a60-9b29-46a8-a318-a3ff1eb48b32-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:30:55.919722 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.919686    5704 generic.go:356] "Generic (PLEG): container finished" podID="e1de5a60-9b29-46a8-a318-a3ff1eb48b32" containerID="a9909badd1b3bcb363a4f9af6719b006571c48fd13e62f72edef1a73ab73c2eb" exitCode=0
Oct 30 19:30:55.919907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.919730    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"
Oct 30 19:30:55.919907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.919760    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw" event={"ID":"e1de5a60-9b29-46a8-a318-a3ff1eb48b32","Type":"ContainerDied","Data":"a9909badd1b3bcb363a4f9af6719b006571c48fd13e62f72edef1a73ab73c2eb"}
Oct 30 19:30:55.919907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.919804    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw" event={"ID":"e1de5a60-9b29-46a8-a318-a3ff1eb48b32","Type":"ContainerDied","Data":"d11ad2ba0b610e5cf33ce1df4d6a58cd0922cd07918355af12c1c35deb89d597"}
Oct 30 19:30:55.919907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.919829    5704 scope.go:122] "RemoveContainer" containerID="a9909badd1b3bcb363a4f9af6719b006571c48fd13e62f72edef1a73ab73c2eb"
Oct 30 19:30:55.929912 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.929891    5704 scope.go:122] "RemoveContainer" containerID="a9909badd1b3bcb363a4f9af6719b006571c48fd13e62f72edef1a73ab73c2eb"
Oct 30 19:30:55.930193 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:30:55.930174    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9909badd1b3bcb363a4f9af6719b006571c48fd13e62f72edef1a73ab73c2eb\": container with ID starting with a9909badd1b3bcb363a4f9af6719b006571c48fd13e62f72edef1a73ab73c2eb not found: ID does not exist" containerID="a9909badd1b3bcb363a4f9af6719b006571c48fd13e62f72edef1a73ab73c2eb"
Oct 30 19:30:55.930254 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:55.930202    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9909badd1b3bcb363a4f9af6719b006571c48fd13e62f72edef1a73ab73c2eb"} err="failed to get container status \"a9909badd1b3bcb363a4f9af6719b006571c48fd13e62f72edef1a73ab73c2eb\": rpc error: code = NotFound desc = could not find container \"a9909badd1b3bcb363a4f9af6719b006571c48fd13e62f72edef1a73ab73c2eb\": container with ID starting with a9909badd1b3bcb363a4f9af6719b006571c48fd13e62f72edef1a73ab73c2eb not found: ID does not exist"
Oct 30 19:30:56.075524 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:56.075494    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"]
Oct 30 19:30:56.121308 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:56.121277    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-4833f3-c4f6895b6-5vbsw"]
Oct 30 19:30:56.900306 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:56.900277    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:57.180314 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:57.180241    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1de5a60-9b29-46a8-a318-a3ff1eb48b32" path="/var/lib/kubelet/pods/e1de5a60-9b29-46a8-a318-a3ff1eb48b32/volumes"
Oct 30 19:30:57.199636 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:57.199594    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:57.384778 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:57.384739    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:57.863998 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:57.863960    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:30:58.667806 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:30:58.667773    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:31:00.192425 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:00.192389    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:01.431754 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:01.431713    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:02.070379 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:02.070347    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:02.300228 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:02.300194    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:02.524285 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:02.524254    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:03.383304 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:31:03.383269    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:31:04.750478 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:04.750440    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:04.944668 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:04.944637    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:05.125306 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:05.125234    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:06.906207 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:06.906170    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:09.172814 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:09.172774    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:11.484649 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:11.484597    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:14.636228 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:14.636189    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:14.825206 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:14.825173    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:16.406173 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:16.406140    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:17.788225 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:17.788194    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:17.978082 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:17.978050    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:21.742256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:21.742205    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:25.306427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:25.306383    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:25.484400 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:25.484364    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:25.672895 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:25.672805    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:27.367787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:27.367755    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:28.397978 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:28.397946    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:32.418951 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:32.418916    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"]
Oct 30 19:31:32.520787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:32.520759    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"]
Oct 30 19:31:32.800559 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:32.800528    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:32.983028 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:32.982989    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:33.242987 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:33.242911    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:33.384278 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:31:33.384238    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:31:33.425596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:33.425565    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:34.198706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:34.198670    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:35.434848 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:35.434820    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz_516e0cbe-fd75-43ba-b635-c3427db43103/payload-processing/0.log"
Oct 30 19:31:35.666292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:35.666270    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz_516e0cbe-fd75-43ba-b635-c3427db43103/payload-processing/0.log"
Oct 30 19:31:37.419486 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.419448    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77" podUID="3e87b294-1fba-42e8-ae62-66e3f401cbbf" containerName="payload-processing" containerID="cri-o://ed623f1c822348ef1b232b7cb8d309b6f0d79e3789e855a6d701609f38833c6c" gracePeriod=25
Oct 30 19:31:37.521830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.521797    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr" podUID="a1466886-3b15-457a-a6c3-c17877af9aa1" containerName="payload-pre-processing" containerID="cri-o://da28bd655ef38cfb9f7f65ff601841d0504f112a5171cfb5140f1d0e0de08077" gracePeriod=25
Oct 30 19:31:37.681844 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.681780    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"
Oct 30 19:31:37.703622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.703592    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e87b294-1fba-42e8-ae62-66e3f401cbbf-kube-api-access-xwpjd\" (UniqueName: \"kubernetes.io/projected/3e87b294-1fba-42e8-ae62-66e3f401cbbf-kube-api-access-xwpjd\") pod \"3e87b294-1fba-42e8-ae62-66e3f401cbbf\" (UID: \"3e87b294-1fba-42e8-ae62-66e3f401cbbf\") "
Oct 30 19:31:37.703827 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.703706    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3e87b294-1fba-42e8-ae62-66e3f401cbbf-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e87b294-1fba-42e8-ae62-66e3f401cbbf-plugins-config-volume\") pod \"3e87b294-1fba-42e8-ae62-66e3f401cbbf\" (UID: \"3e87b294-1fba-42e8-ae62-66e3f401cbbf\") "
Oct 30 19:31:37.704365 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.704338    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e87b294-1fba-42e8-ae62-66e3f401cbbf-plugins-config-volume" pod "3e87b294-1fba-42e8-ae62-66e3f401cbbf" (UID: "3e87b294-1fba-42e8-ae62-66e3f401cbbf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:31:37.706575 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.706546    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e87b294-1fba-42e8-ae62-66e3f401cbbf-kube-api-access-xwpjd" pod "3e87b294-1fba-42e8-ae62-66e3f401cbbf" (UID: "3e87b294-1fba-42e8-ae62-66e3f401cbbf"). InnerVolumeSpecName "kube-api-access-xwpjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:37.767009 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.766986    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"
Oct 30 19:31:37.805388 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.805352    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1466886-3b15-457a-a6c3-c17877af9aa1-kube-api-access-w5b6w\" (UniqueName: \"kubernetes.io/projected/a1466886-3b15-457a-a6c3-c17877af9aa1-kube-api-access-w5b6w\") pod \"a1466886-3b15-457a-a6c3-c17877af9aa1\" (UID: \"a1466886-3b15-457a-a6c3-c17877af9aa1\") "
Oct 30 19:31:37.805606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.805585    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1466886-3b15-457a-a6c3-c17877af9aa1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1466886-3b15-457a-a6c3-c17877af9aa1-plugins-config-volume\") pod \"a1466886-3b15-457a-a6c3-c17877af9aa1\" (UID: \"a1466886-3b15-457a-a6c3-c17877af9aa1\") "
Oct 30 19:31:37.806001 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.805979    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3e87b294-1fba-42e8-ae62-66e3f401cbbf-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:37.806128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.805991    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1466886-3b15-457a-a6c3-c17877af9aa1-plugins-config-volume" pod "a1466886-3b15-457a-a6c3-c17877af9aa1" (UID: "a1466886-3b15-457a-a6c3-c17877af9aa1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:31:37.806128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.806002    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwpjd\" (UniqueName: \"kubernetes.io/projected/3e87b294-1fba-42e8-ae62-66e3f401cbbf-kube-api-access-xwpjd\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:37.807571 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.807544    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1466886-3b15-457a-a6c3-c17877af9aa1-kube-api-access-w5b6w" pod "a1466886-3b15-457a-a6c3-c17877af9aa1" (UID: "a1466886-3b15-457a-a6c3-c17877af9aa1"). InnerVolumeSpecName "kube-api-access-w5b6w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:31:37.907700 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.907670    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5b6w\" (UniqueName: \"kubernetes.io/projected/a1466886-3b15-457a-a6c3-c17877af9aa1-kube-api-access-w5b6w\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:37.907700 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:37.907703    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a1466886-3b15-457a-a6c3-c17877af9aa1-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:31:38.171103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.171065    5704 generic.go:356] "Generic (PLEG): container finished" podID="3e87b294-1fba-42e8-ae62-66e3f401cbbf" containerID="ed623f1c822348ef1b232b7cb8d309b6f0d79e3789e855a6d701609f38833c6c" exitCode=0
Oct 30 19:31:38.171103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.171100    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"
Oct 30 19:31:38.171357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.171133    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77" event={"ID":"3e87b294-1fba-42e8-ae62-66e3f401cbbf","Type":"ContainerDied","Data":"ed623f1c822348ef1b232b7cb8d309b6f0d79e3789e855a6d701609f38833c6c"}
Oct 30 19:31:38.171357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.171189    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77" event={"ID":"3e87b294-1fba-42e8-ae62-66e3f401cbbf","Type":"ContainerDied","Data":"4266ee74a2b8234ce5767bef44a61848d67fb24ef2c48fb05459f13c566bc33d"}
Oct 30 19:31:38.171357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.171218    5704 scope.go:122] "RemoveContainer" containerID="ed623f1c822348ef1b232b7cb8d309b6f0d79e3789e855a6d701609f38833c6c"
Oct 30 19:31:38.172706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.172683    5704 generic.go:356] "Generic (PLEG): container finished" podID="a1466886-3b15-457a-a6c3-c17877af9aa1" containerID="da28bd655ef38cfb9f7f65ff601841d0504f112a5171cfb5140f1d0e0de08077" exitCode=0
Oct 30 19:31:38.172828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.172737    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"
Oct 30 19:31:38.172828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.172741    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr" event={"ID":"a1466886-3b15-457a-a6c3-c17877af9aa1","Type":"ContainerDied","Data":"da28bd655ef38cfb9f7f65ff601841d0504f112a5171cfb5140f1d0e0de08077"}
Oct 30 19:31:38.172828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.172782    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr" event={"ID":"a1466886-3b15-457a-a6c3-c17877af9aa1","Type":"ContainerDied","Data":"06cf3d72618741dd0dc980d274caec27c46ca11ce34890bd9ab6fe3cdb533e09"}
Oct 30 19:31:38.181820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.181793    5704 scope.go:122] "RemoveContainer" containerID="ed623f1c822348ef1b232b7cb8d309b6f0d79e3789e855a6d701609f38833c6c"
Oct 30 19:31:38.182095 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:31:38.182075    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed623f1c822348ef1b232b7cb8d309b6f0d79e3789e855a6d701609f38833c6c\": container with ID starting with ed623f1c822348ef1b232b7cb8d309b6f0d79e3789e855a6d701609f38833c6c not found: ID does not exist" containerID="ed623f1c822348ef1b232b7cb8d309b6f0d79e3789e855a6d701609f38833c6c"
Oct 30 19:31:38.182187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.182111    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed623f1c822348ef1b232b7cb8d309b6f0d79e3789e855a6d701609f38833c6c"} err="failed to get container status \"ed623f1c822348ef1b232b7cb8d309b6f0d79e3789e855a6d701609f38833c6c\": rpc error: code = NotFound desc = could not find container \"ed623f1c822348ef1b232b7cb8d309b6f0d79e3789e855a6d701609f38833c6c\": container with ID starting with ed623f1c822348ef1b232b7cb8d309b6f0d79e3789e855a6d701609f38833c6c not found: ID does not exist"
Oct 30 19:31:38.182187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.182128    5704 scope.go:122] "RemoveContainer" containerID="da28bd655ef38cfb9f7f65ff601841d0504f112a5171cfb5140f1d0e0de08077"
Oct 30 19:31:38.191657 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.191640    5704 scope.go:122] "RemoveContainer" containerID="da28bd655ef38cfb9f7f65ff601841d0504f112a5171cfb5140f1d0e0de08077"
Oct 30 19:31:38.191902 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:31:38.191882    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da28bd655ef38cfb9f7f65ff601841d0504f112a5171cfb5140f1d0e0de08077\": container with ID starting with da28bd655ef38cfb9f7f65ff601841d0504f112a5171cfb5140f1d0e0de08077 not found: ID does not exist" containerID="da28bd655ef38cfb9f7f65ff601841d0504f112a5171cfb5140f1d0e0de08077"
Oct 30 19:31:38.191945 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.191910    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da28bd655ef38cfb9f7f65ff601841d0504f112a5171cfb5140f1d0e0de08077"} err="failed to get container status \"da28bd655ef38cfb9f7f65ff601841d0504f112a5171cfb5140f1d0e0de08077\": rpc error: code = NotFound desc = could not find container \"da28bd655ef38cfb9f7f65ff601841d0504f112a5171cfb5140f1d0e0de08077\": container with ID starting with da28bd655ef38cfb9f7f65ff601841d0504f112a5171cfb5140f1d0e0de08077 not found: ID does not exist"
Oct 30 19:31:38.281635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.281600    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:38.327046 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.326999    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"]
Oct 30 19:31:38.331201 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.331173    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-b-c16ba7-f8ddb9787-rrxvr"]
Oct 30 19:31:38.433105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.433005    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"]
Oct 30 19:31:38.437682 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:38.437655    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-b-c16ba7-6c64ccf8cd-79w77"]
Oct 30 19:31:39.180258 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:39.180225    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e87b294-1fba-42e8-ae62-66e3f401cbbf" path="/var/lib/kubelet/pods/3e87b294-1fba-42e8-ae62-66e3f401cbbf/volumes"
Oct 30 19:31:39.180614 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:39.180601    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1466886-3b15-457a-a6c3-c17877af9aa1" path="/var/lib/kubelet/pods/a1466886-3b15-457a-a6c3-c17877af9aa1/volumes"
Oct 30 19:31:40.609026 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:40.608995    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:31:43.420528 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:43.420440    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:48.207344 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:48.207311    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:48.438850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:48.438814    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:48.613260 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:48.613223    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:52.983288 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:52.983246    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:31:57.568003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:31:57.567957    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:02.290875 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:02.290844    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:03.383842 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:32:03.383806    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:32:06.274508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:06.274473    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:07.824810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:07.824770    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:08.092641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:08.092578    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:08.278337 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:08.278300    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:08.478550 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:08.478463    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:08.759324 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:08.759290    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:09.549235 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:09.549197    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:12.994847 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:12.994810    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:13.184002 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:13.183970    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:13.365638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:13.365605    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:15.392675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:15.392635    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:15.579274 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:15.579233    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:15.768083 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:15.768028    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:17.701209 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:17.701172    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:17.979027 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:17.978925    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:21.421087 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:21.421057    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:21.905786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:21.905751    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:22.988077 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.988044    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"]
Oct 30 19:32:22.988623 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.988608    5704 cpu_manager.go:414] "containerMap: removing container" podUID="3e87b294-1fba-42e8-ae62-66e3f401cbbf" containerName="payload-processing"
Oct 30 19:32:22.988682 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.988624    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e87b294-1fba-42e8-ae62-66e3f401cbbf" containerName="payload-processing"
Oct 30 19:32:22.988682 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.988659    5704 cpu_manager.go:414] "containerMap: removing container" podUID="a1466886-3b15-457a-a6c3-c17877af9aa1" containerName="payload-pre-processing"
Oct 30 19:32:22.988682 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.988664    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1466886-3b15-457a-a6c3-c17877af9aa1" containerName="payload-pre-processing"
Oct 30 19:32:22.988682 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.988678    5704 cpu_manager.go:414] "containerMap: removing container" podUID="702caeab-5ddb-4a78-b1a2-80993766a989" containerName="payload-pre-processing"
Oct 30 19:32:22.988810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.988683    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="702caeab-5ddb-4a78-b1a2-80993766a989" containerName="payload-pre-processing"
Oct 30 19:32:22.988810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.988694    5704 cpu_manager.go:414] "containerMap: removing container" podUID="e1de5a60-9b29-46a8-a318-a3ff1eb48b32" containerName="maas-api"
Oct 30 19:32:22.988810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.988699    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1de5a60-9b29-46a8-a318-a3ff1eb48b32" containerName="maas-api"
Oct 30 19:32:22.988810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.988786    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="702caeab-5ddb-4a78-b1a2-80993766a989" containerName="payload-pre-processing"
Oct 30 19:32:22.988810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.988796    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1de5a60-9b29-46a8-a318-a3ff1eb48b32" containerName="maas-api"
Oct 30 19:32:22.988810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.988806    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e87b294-1fba-42e8-ae62-66e3f401cbbf" containerName="payload-processing"
Oct 30 19:32:22.989013 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.988815    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1466886-3b15-457a-a6c3-c17877af9aa1" containerName="payload-pre-processing"
Oct 30 19:32:22.993469 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:22.993449    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"
Oct 30 19:32:23.043177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:23.043150    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-f3b5af\""
Oct 30 19:32:23.089369 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:23.089333    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-f3b5af-dockercfg-4dvwv\""
Oct 30 19:32:23.091573 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:23.091547    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"]
Oct 30 19:32:23.100596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:23.100573    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxc57\" (UniqueName: \"kubernetes.io/projected/769034b9-5ad4-46e5-be15-b79d307aa077-kube-api-access-pxc57\") pod \"payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b\" (UID: \"769034b9-5ad4-46e5-be15-b79d307aa077\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"
Oct 30 19:32:23.100686 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:23.100623    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/769034b9-5ad4-46e5-be15-b79d307aa077-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b\" (UID: \"769034b9-5ad4-46e5-be15-b79d307aa077\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"
Oct 30 19:32:23.201857 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:23.201831    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pxc57\" (UniqueName: \"kubernetes.io/projected/769034b9-5ad4-46e5-be15-b79d307aa077-kube-api-access-pxc57\") pod \"payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b\" (UID: \"769034b9-5ad4-46e5-be15-b79d307aa077\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"
Oct 30 19:32:23.201998 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:23.201887    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/769034b9-5ad4-46e5-be15-b79d307aa077-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b\" (UID: \"769034b9-5ad4-46e5-be15-b79d307aa077\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"
Oct 30 19:32:23.202839 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:23.202817    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/769034b9-5ad4-46e5-be15-b79d307aa077-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b\" (UID: \"769034b9-5ad4-46e5-be15-b79d307aa077\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"
Oct 30 19:32:23.259286 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:23.259255    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxc57\" (UniqueName: \"kubernetes.io/projected/769034b9-5ad4-46e5-be15-b79d307aa077-kube-api-access-pxc57\") pod \"payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b\" (UID: \"769034b9-5ad4-46e5-be15-b79d307aa077\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"
Oct 30 19:32:23.307282 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:23.307259    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"
Oct 30 19:32:23.679821 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:23.679742    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"]
Oct 30 19:32:23.686857 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:32:23.686828    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod769034b9_5ad4_46e5_be15_b79d307aa077.slice/crio-ebeba1ecfd650a18e5075bf572b058f893bfe182c1ab226fb69c4144afdff60d WatchSource:0}: Error finding container ebeba1ecfd650a18e5075bf572b058f893bfe182c1ab226fb69c4144afdff60d: Status 404 returned error can't find the container with id ebeba1ecfd650a18e5075bf572b058f893bfe182c1ab226fb69c4144afdff60d
Oct 30 19:32:24.476596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:24.476565    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b" event={"ID":"769034b9-5ad4-46e5-be15-b79d307aa077","Type":"ContainerStarted","Data":"200d0de2d52a7a68f62131bc88836cd8d9c270a77e564d96c0df897855815627"}
Oct 30 19:32:24.476967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:24.476618    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b" event={"ID":"769034b9-5ad4-46e5-be15-b79d307aa077","Type":"ContainerStarted","Data":"ebeba1ecfd650a18e5075bf572b058f893bfe182c1ab226fb69c4144afdff60d"}
Oct 30 19:32:24.476967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:24.476646    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"
Oct 30 19:32:24.573766 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:24.573739    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:28.743518 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:28.743471    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b" podStartSLOduration=6.74345713 podStartE2EDuration="6.74345713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:24.596244194 +0000 UTC m=+2717.542412098" watchObservedRunningTime="2026-10-30 19:32:28.74345713 +0000 UTC m=+2721.689625062"
Oct 30 19:32:28.743915 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:28.743786    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:32:30.796294 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:30.796255    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:33.383158 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:32:33.383125    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:32:35.395056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:35.395003    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:35.484227 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:35.484193    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"
Oct 30 19:32:38.536622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:38.536587    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:39.164150 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:39.164101    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:39.464864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:39.464779    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:39.985429 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:39.985397    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:44.707168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:44.707133    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:49.401431 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:49.401392    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:51.764998 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:51.764961    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:52.013681 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.013641    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:52.120232 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.120157    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"]
Oct 30 19:32:52.124286 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.124263    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.219756 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.219728    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-330025-openshift-default-dockercfg-9qb6q\""
Oct 30 19:32:52.224193 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.224162    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.224314 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.224283    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.224420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.224406    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.224588 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.224568    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.224650 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.224618    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.224650 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.224643    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.224748 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.224673    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.224810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.224794    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z594b\" (UniqueName: \"kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-kube-api-access-z594b\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.224863 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.224834    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.268459 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.268431    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"]
Oct 30 19:32:52.325997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.325952    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z594b\" (UniqueName: \"kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-kube-api-access-z594b\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.326194 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.326017    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.326194 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.326087    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.326194 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.326142    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.326194 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.326182    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.326416 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.326233    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.326641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.326610    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.326641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.326626    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.326791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.326261    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.326791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.326680    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.326791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.326687    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.326791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.326742    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.327023 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.326999    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.327190 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.327172    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.328537 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.328516    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.328644 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.328573    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.406682 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.406609    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.406682 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.406649    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z594b\" (UniqueName: \"kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-kube-api-access-z594b\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.437907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.437830    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:52.870624 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.870594    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"]
Oct 30 19:32:52.877295 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:32:52.877271    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4171f5d_d080_40d0_8a48_f592ba37f8d6.slice/crio-165e1a86b0bf4fe048fe0339c680d371e686ed6675c9d9911fedc015a2773670 WatchSource:0}: Error finding container 165e1a86b0bf4fe048fe0339c680d371e686ed6675c9d9911fedc015a2773670: Status 404 returned error can't find the container with id 165e1a86b0bf4fe048fe0339c680d371e686ed6675c9d9911fedc015a2773670
Oct 30 19:32:52.879717 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.879682    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:32:52.879803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.879751    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:32:52.879803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:52.879784    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:32:53.669055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:53.668100    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"]
Oct 30 19:32:53.669055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:53.668362    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74" podUID="5357c9cd-0b77-44e6-930a-2b1f26884f9a" containerName="maas-api" containerID="cri-o://d8c9b10539368e0dada0e94aa76c573f2e785a6e658d05dc596936cbc79dc0f7" gracePeriod=30
Oct 30 19:32:53.681262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:53.681226    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp" event={"ID":"d4171f5d-d080-40d0-8a48-f592ba37f8d6","Type":"ContainerStarted","Data":"efe220f8c7a9f76a58881e0f10e11b7868c279c5041455ac57c531d9cee9cf69"}
Oct 30 19:32:53.681499 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:53.681475    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp" event={"ID":"d4171f5d-d080-40d0-8a48-f592ba37f8d6","Type":"ContainerStarted","Data":"165e1a86b0bf4fe048fe0339c680d371e686ed6675c9d9911fedc015a2773670"}
Oct 30 19:32:53.804190 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:53.804158    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:53.965841 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:53.965801    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:32:54.044315 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.041249    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp" podStartSLOduration=2.041228306 podStartE2EDuration="2.041228306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:32:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:32:53.993503156 +0000 UTC m=+2746.939671060" watchObservedRunningTime="2026-10-30 19:32:54.041228306 +0000 UTC m=+2746.987396208"
Oct 30 19:32:54.050670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.050641    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5357c9cd-0b77-44e6-930a-2b1f26884f9a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5357c9cd-0b77-44e6-930a-2b1f26884f9a-openshift-service-ca\") pod \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") "
Oct 30 19:32:54.050840 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.050769    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-maas-api-tls\") pod \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") "
Oct 30 19:32:54.050840 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.050807    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5357c9cd-0b77-44e6-930a-2b1f26884f9a-kube-api-access-d9425\" (UniqueName: \"kubernetes.io/projected/5357c9cd-0b77-44e6-930a-2b1f26884f9a-kube-api-access-d9425\") pod \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") "
Oct 30 19:32:54.050914 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.050895    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-metrics-certs\") pod \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\" (UID: \"5357c9cd-0b77-44e6-930a-2b1f26884f9a\") "
Oct 30 19:32:54.051071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.051010    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5357c9cd-0b77-44e6-930a-2b1f26884f9a-openshift-service-ca" pod "5357c9cd-0b77-44e6-930a-2b1f26884f9a" (UID: "5357c9cd-0b77-44e6-930a-2b1f26884f9a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:32:54.051395 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.051372    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5357c9cd-0b77-44e6-930a-2b1f26884f9a-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:54.053108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.053085    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-maas-api-tls" pod "5357c9cd-0b77-44e6-930a-2b1f26884f9a" (UID: "5357c9cd-0b77-44e6-930a-2b1f26884f9a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:54.053197 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.053086    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5357c9cd-0b77-44e6-930a-2b1f26884f9a-kube-api-access-d9425" pod "5357c9cd-0b77-44e6-930a-2b1f26884f9a" (UID: "5357c9cd-0b77-44e6-930a-2b1f26884f9a"). InnerVolumeSpecName "kube-api-access-d9425". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:32:54.053197 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.053136    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-metrics-certs" pod "5357c9cd-0b77-44e6-930a-2b1f26884f9a" (UID: "5357c9cd-0b77-44e6-930a-2b1f26884f9a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:32:54.152396 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.152366    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:54.152396 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.152394    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9425\" (UniqueName: \"kubernetes.io/projected/5357c9cd-0b77-44e6-930a-2b1f26884f9a-kube-api-access-d9425\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:54.152396 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.152405    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5357c9cd-0b77-44e6-930a-2b1f26884f9a-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:32:54.349728 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.349628    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:54.444213 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.444164    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:32:54.444213 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.444207    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 19:32:54.444483 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.444266    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp" podUID="d4171f5d-d080-40d0-8a48-f592ba37f8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 19:32:54.578723 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.578687    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:54.689825 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.689745    5704 generic.go:356] "Generic (PLEG): container finished" podID="5357c9cd-0b77-44e6-930a-2b1f26884f9a" containerID="d8c9b10539368e0dada0e94aa76c573f2e785a6e658d05dc596936cbc79dc0f7" exitCode=0
Oct 30 19:32:54.689825 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.689807    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"
Oct 30 19:32:54.690086 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.689832    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74" event={"ID":"5357c9cd-0b77-44e6-930a-2b1f26884f9a","Type":"ContainerDied","Data":"d8c9b10539368e0dada0e94aa76c573f2e785a6e658d05dc596936cbc79dc0f7"}
Oct 30 19:32:54.690086 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.689899    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74" event={"ID":"5357c9cd-0b77-44e6-930a-2b1f26884f9a","Type":"ContainerDied","Data":"3a93bd0f70e64f66c4f04b04ebd42fd53b8b8b75c7575b23e9a8292c2d822b85"}
Oct 30 19:32:54.690086 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.689932    5704 scope.go:122] "RemoveContainer" containerID="d8c9b10539368e0dada0e94aa76c573f2e785a6e658d05dc596936cbc79dc0f7"
Oct 30 19:32:54.781548 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.781526    5704 scope.go:122] "RemoveContainer" containerID="d8c9b10539368e0dada0e94aa76c573f2e785a6e658d05dc596936cbc79dc0f7"
Oct 30 19:32:54.781997 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:32:54.781974    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8c9b10539368e0dada0e94aa76c573f2e785a6e658d05dc596936cbc79dc0f7\": container with ID starting with d8c9b10539368e0dada0e94aa76c573f2e785a6e658d05dc596936cbc79dc0f7 not found: ID does not exist" containerID="d8c9b10539368e0dada0e94aa76c573f2e785a6e658d05dc596936cbc79dc0f7"
Oct 30 19:32:54.782092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.782006    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8c9b10539368e0dada0e94aa76c573f2e785a6e658d05dc596936cbc79dc0f7"} err="failed to get container status \"d8c9b10539368e0dada0e94aa76c573f2e785a6e658d05dc596936cbc79dc0f7\": rpc error: code = NotFound desc = could not find container \"d8c9b10539368e0dada0e94aa76c573f2e785a6e658d05dc596936cbc79dc0f7\": container with ID starting with d8c9b10539368e0dada0e94aa76c573f2e785a6e658d05dc596936cbc79dc0f7 not found: ID does not exist"
Oct 30 19:32:54.848143 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.848062    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"]
Oct 30 19:32:54.893136 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:54.893107    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-7a790b-684b4587b9-fgm74"]
Oct 30 19:32:55.179347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:55.179317    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5357c9cd-0b77-44e6-930a-2b1f26884f9a" path="/var/lib/kubelet/pods/5357c9cd-0b77-44e6-930a-2b1f26884f9a/volumes"
Oct 30 19:32:55.444242 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:55.444156    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 19:32:55.444242 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:55.444222    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp" podUID="d4171f5d-d080-40d0-8a48-f592ba37f8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 19:32:55.805144 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:55.805108    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:56.089628 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:56.089523    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:56.438727 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:56.438635    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 19:32:56.439318 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:56.439288    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp" podUID="d4171f5d-d080-40d0-8a48-f592ba37f8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 19:32:56.439814 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:56.439013    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:57.068551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:57.068516    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:57.440112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:57.439356    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 19:32:57.440112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:57.439421    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp" podUID="d4171f5d-d080-40d0-8a48-f592ba37f8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 19:32:58.059896 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:58.059852    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:58.444225 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:58.444137    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 19:32:58.444225 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:58.444195    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp" podUID="d4171f5d-d080-40d0-8a48-f592ba37f8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 19:32:58.947291 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:58.947252    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:32:59.253944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:59.253907    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:59.435614 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:59.435585    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:32:59.438646 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:59.438594    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused" start-of-body=
Oct 30 19:32:59.438853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:32:59.438825    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp" podUID="d4171f5d-d080-40d0-8a48-f592ba37f8d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.137:15021/healthz/ready\": dial tcp 10.134.0.137:15021: connect: connection refused"
Oct 30 19:33:00.682294 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:00.682256    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:00.890534 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:00.890498    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:33:00.890979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:00.890942    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:33:00.891971 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:00.891952    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 19:33:02.105298 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:02.105265    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:03.384255 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:33:03.384214    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:33:05.685132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:05.685073    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:09.879664 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:09.879632    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:10.390953 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:10.390911    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:14.943877 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:14.943837    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:19.896666 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:19.893851    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:22.333404 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:22.333366    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:26.850048 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:26.849999    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:32.302338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:32.302302    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:32.433533 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:32.429467    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:33:32.535989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:32.535958    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:33.184256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:33.181632    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:33.388217 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:33:33.388168    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:33:37.861899 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:37.861859    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:38.959589 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:38.959558    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"]
Oct 30 19:33:39.059904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:39.059874    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"]
Oct 30 19:33:39.352458 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:39.352382    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:39.531274 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:39.531235    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:39.743816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:39.738307    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:40.586273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:40.586230    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:41.923542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:41.923509    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:43.364010 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:43.363973    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:43.593906 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:43.593879    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:43.785616 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:43.785567    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:43.959992 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:43.959938    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292" podUID="82681f31-bc24-4592-8f1f-5ea5b852e40a" containerName="payload-processing" containerID="cri-o://36f3c4a69e89a655f8e14238836540e1d839c80f520c3b6592361b69f07bc1cf" gracePeriod=25
Oct 30 19:33:44.060546 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.060487    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj" podUID="020fbb14-fd8e-4941-9c3f-91309c71ce47" containerName="payload-pre-processing" containerID="cri-o://c805cd3852f9317f23e53c0454d4e8d71bc469e7f2667bd7e65ba6aa64adcb09" gracePeriod=25
Oct 30 19:33:44.228201 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.228170    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"
Oct 30 19:33:44.325197 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.325174    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"
Oct 30 19:33:44.364951 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.364917    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/82681f31-bc24-4592-8f1f-5ea5b852e40a-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82681f31-bc24-4592-8f1f-5ea5b852e40a-plugins-config-volume\") pod \"82681f31-bc24-4592-8f1f-5ea5b852e40a\" (UID: \"82681f31-bc24-4592-8f1f-5ea5b852e40a\") "
Oct 30 19:33:44.365424 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.365024    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/82681f31-bc24-4592-8f1f-5ea5b852e40a-kube-api-access-lrkdv\" (UniqueName: \"kubernetes.io/projected/82681f31-bc24-4592-8f1f-5ea5b852e40a-kube-api-access-lrkdv\") pod \"82681f31-bc24-4592-8f1f-5ea5b852e40a\" (UID: \"82681f31-bc24-4592-8f1f-5ea5b852e40a\") "
Oct 30 19:33:44.365424 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.365370    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82681f31-bc24-4592-8f1f-5ea5b852e40a-plugins-config-volume" pod "82681f31-bc24-4592-8f1f-5ea5b852e40a" (UID: "82681f31-bc24-4592-8f1f-5ea5b852e40a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:44.365621 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.365601    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82681f31-bc24-4592-8f1f-5ea5b852e40a-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:44.367064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.367021    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82681f31-bc24-4592-8f1f-5ea5b852e40a-kube-api-access-lrkdv" pod "82681f31-bc24-4592-8f1f-5ea5b852e40a" (UID: "82681f31-bc24-4592-8f1f-5ea5b852e40a"). InnerVolumeSpecName "kube-api-access-lrkdv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:44.465994 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.465971    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/020fbb14-fd8e-4941-9c3f-91309c71ce47-kube-api-access-xsrrd\" (UniqueName: \"kubernetes.io/projected/020fbb14-fd8e-4941-9c3f-91309c71ce47-kube-api-access-xsrrd\") pod \"020fbb14-fd8e-4941-9c3f-91309c71ce47\" (UID: \"020fbb14-fd8e-4941-9c3f-91309c71ce47\") "
Oct 30 19:33:44.466171 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.466105    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/020fbb14-fd8e-4941-9c3f-91309c71ce47-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/020fbb14-fd8e-4941-9c3f-91309c71ce47-plugins-config-volume\") pod \"020fbb14-fd8e-4941-9c3f-91309c71ce47\" (UID: \"020fbb14-fd8e-4941-9c3f-91309c71ce47\") "
Oct 30 19:33:44.466440 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.466417    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/020fbb14-fd8e-4941-9c3f-91309c71ce47-plugins-config-volume" pod "020fbb14-fd8e-4941-9c3f-91309c71ce47" (UID: "020fbb14-fd8e-4941-9c3f-91309c71ce47"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:44.466521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.466508    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/020fbb14-fd8e-4941-9c3f-91309c71ce47-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:44.466577 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.466528    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrkdv\" (UniqueName: \"kubernetes.io/projected/82681f31-bc24-4592-8f1f-5ea5b852e40a-kube-api-access-lrkdv\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:44.467888 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.467870    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/020fbb14-fd8e-4941-9c3f-91309c71ce47-kube-api-access-xsrrd" pod "020fbb14-fd8e-4941-9c3f-91309c71ce47" (UID: "020fbb14-fd8e-4941-9c3f-91309c71ce47"). InnerVolumeSpecName "kube-api-access-xsrrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:44.567732 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.567658    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsrrd\" (UniqueName: \"kubernetes.io/projected/020fbb14-fd8e-4941-9c3f-91309c71ce47-kube-api-access-xsrrd\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:44.991945 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.991914    5704 generic.go:356] "Generic (PLEG): container finished" podID="82681f31-bc24-4592-8f1f-5ea5b852e40a" containerID="36f3c4a69e89a655f8e14238836540e1d839c80f520c3b6592361b69f07bc1cf" exitCode=0
Oct 30 19:33:44.992101 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.991962    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"
Oct 30 19:33:44.992101 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.991999    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292" event={"ID":"82681f31-bc24-4592-8f1f-5ea5b852e40a","Type":"ContainerDied","Data":"36f3c4a69e89a655f8e14238836540e1d839c80f520c3b6592361b69f07bc1cf"}
Oct 30 19:33:44.992101 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.992067    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292" event={"ID":"82681f31-bc24-4592-8f1f-5ea5b852e40a","Type":"ContainerDied","Data":"d5ea9e8649b597f476af8b9aa7b1fb5b4de9695224615a274139533f7cc4f547"}
Oct 30 19:33:44.992101 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.992091    5704 scope.go:122] "RemoveContainer" containerID="36f3c4a69e89a655f8e14238836540e1d839c80f520c3b6592361b69f07bc1cf"
Oct 30 19:33:44.993622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.993602    5704 generic.go:356] "Generic (PLEG): container finished" podID="020fbb14-fd8e-4941-9c3f-91309c71ce47" containerID="c805cd3852f9317f23e53c0454d4e8d71bc469e7f2667bd7e65ba6aa64adcb09" exitCode=0
Oct 30 19:33:44.993715 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.993663    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj" event={"ID":"020fbb14-fd8e-4941-9c3f-91309c71ce47","Type":"ContainerDied","Data":"c805cd3852f9317f23e53c0454d4e8d71bc469e7f2667bd7e65ba6aa64adcb09"}
Oct 30 19:33:44.993779 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.993723    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj" event={"ID":"020fbb14-fd8e-4941-9c3f-91309c71ce47","Type":"ContainerDied","Data":"16441d7f158b57d0f95ba3e05ff6c773cdb26566e057c3d260205a05476d96e1"}
Oct 30 19:33:44.993779 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:44.993670    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"
Oct 30 19:33:45.004559 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:45.004539    5704 scope.go:122] "RemoveContainer" containerID="36f3c4a69e89a655f8e14238836540e1d839c80f520c3b6592361b69f07bc1cf"
Oct 30 19:33:45.004830 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:33:45.004814    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"36f3c4a69e89a655f8e14238836540e1d839c80f520c3b6592361b69f07bc1cf\": container with ID starting with 36f3c4a69e89a655f8e14238836540e1d839c80f520c3b6592361b69f07bc1cf not found: ID does not exist" containerID="36f3c4a69e89a655f8e14238836540e1d839c80f520c3b6592361b69f07bc1cf"
Oct 30 19:33:45.004882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:45.004838    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"36f3c4a69e89a655f8e14238836540e1d839c80f520c3b6592361b69f07bc1cf"} err="failed to get container status \"36f3c4a69e89a655f8e14238836540e1d839c80f520c3b6592361b69f07bc1cf\": rpc error: code = NotFound desc = could not find container \"36f3c4a69e89a655f8e14238836540e1d839c80f520c3b6592361b69f07bc1cf\": container with ID starting with 36f3c4a69e89a655f8e14238836540e1d839c80f520c3b6592361b69f07bc1cf not found: ID does not exist"
Oct 30 19:33:45.004882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:45.004861    5704 scope.go:122] "RemoveContainer" containerID="c805cd3852f9317f23e53c0454d4e8d71bc469e7f2667bd7e65ba6aa64adcb09"
Oct 30 19:33:45.013823 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:45.013804    5704 scope.go:122] "RemoveContainer" containerID="c805cd3852f9317f23e53c0454d4e8d71bc469e7f2667bd7e65ba6aa64adcb09"
Oct 30 19:33:45.014097 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:33:45.014069    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c805cd3852f9317f23e53c0454d4e8d71bc469e7f2667bd7e65ba6aa64adcb09\": container with ID starting with c805cd3852f9317f23e53c0454d4e8d71bc469e7f2667bd7e65ba6aa64adcb09 not found: ID does not exist" containerID="c805cd3852f9317f23e53c0454d4e8d71bc469e7f2667bd7e65ba6aa64adcb09"
Oct 30 19:33:45.014157 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:45.014095    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c805cd3852f9317f23e53c0454d4e8d71bc469e7f2667bd7e65ba6aa64adcb09"} err="failed to get container status \"c805cd3852f9317f23e53c0454d4e8d71bc469e7f2667bd7e65ba6aa64adcb09\": rpc error: code = NotFound desc = could not find container \"c805cd3852f9317f23e53c0454d4e8d71bc469e7f2667bd7e65ba6aa64adcb09\": container with ID starting with c805cd3852f9317f23e53c0454d4e8d71bc469e7f2667bd7e65ba6aa64adcb09 not found: ID does not exist"
Oct 30 19:33:45.153041 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:45.153002    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"]
Oct 30 19:33:45.172289 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:45.172254    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292" podUID="82681f31-bc24-4592-8f1f-5ea5b852e40a" containerName="payload-processing" probeResult="failure" output="timeout: failed to connect service \"10.134.0.133:50052\" within 1s: context deadline exceeded"
Oct 30 19:33:45.172412 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:33:45.172337    5704 logging.go:55] [core] [Channel #3964 SubChannel #3965]grpc: addrConn.createTransport failed to connect to {Addr: "10.134.0.133:50052", ServerName: "10.134.0.133:50052", BalancerAttributes: {"<%!p(pickfirstleaf.managedByPickfirstKeyType={})>": "<%!p(bool=true)>" }}. Err: connection error: desc = "transport: Error while dialing: dial tcp 10.134.0.133:50052: operation was canceled"
Oct 30 19:33:45.199133 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:45.199107    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-praxis-a-441836-6684987b75-n2hlj"]
Oct 30 19:33:45.258168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:45.258144    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"]
Oct 30 19:33:45.271068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:45.271045    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-441836-7b4cc5f464-cn292"]
Oct 30 19:33:46.853173 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:46.853136    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:47.099951 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:47.099920    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:47.183124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:47.183020    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="020fbb14-fd8e-4941-9c3f-91309c71ce47" path="/var/lib/kubelet/pods/020fbb14-fd8e-4941-9c3f-91309c71ce47/volumes"
Oct 30 19:33:47.183629 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:47.183611    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="82681f31-bc24-4592-8f1f-5ea5b852e40a" path="/var/lib/kubelet/pods/82681f31-bc24-4592-8f1f-5ea5b852e40a/volumes"
Oct 30 19:33:47.307333 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:47.307294    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:48.634245 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:48.634167    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:53.134838 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:53.134803    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:53.368511 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:53.368478    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:54.267294 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:54.267264    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:57.212897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.212864    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"]
Oct 30 19:33:57.213331 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.213086    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc" podUID="e44e6675-761e-427f-b3b5-4ccbd2bb0dbc" containerName="maas-api" containerID="cri-o://161040bcd6004fff71dfd0ed14f9679328d80fbc65bf3daa5cfd9459b75b0972" gracePeriod=30
Oct 30 19:33:57.363014 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.362976    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:57.498179 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.498157    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:33:57.625526 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.625497    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-openshift-service-ca\") pod \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") "
Oct 30 19:33:57.625732 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.625577    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-metrics-certs\") pod \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") "
Oct 30 19:33:57.625732 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.625687    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-kube-api-access-mz66v\" (UniqueName: \"kubernetes.io/projected/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-kube-api-access-mz66v\") pod \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") "
Oct 30 19:33:57.625902 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.625877    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-openshift-service-ca" pod "e44e6675-761e-427f-b3b5-4ccbd2bb0dbc" (UID: "e44e6675-761e-427f-b3b5-4ccbd2bb0dbc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:33:57.625978 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.625958    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-maas-api-tls\") pod \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\" (UID: \"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc\") "
Oct 30 19:33:57.626525 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.626502    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:57.627787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.627761    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-metrics-certs" pod "e44e6675-761e-427f-b3b5-4ccbd2bb0dbc" (UID: "e44e6675-761e-427f-b3b5-4ccbd2bb0dbc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:57.628311 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.628289    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-maas-api-tls" pod "e44e6675-761e-427f-b3b5-4ccbd2bb0dbc" (UID: "e44e6675-761e-427f-b3b5-4ccbd2bb0dbc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:33:57.628388 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.628312    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-kube-api-access-mz66v" pod "e44e6675-761e-427f-b3b5-4ccbd2bb0dbc" (UID: "e44e6675-761e-427f-b3b5-4ccbd2bb0dbc"). InnerVolumeSpecName "kube-api-access-mz66v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:33:57.727865 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.727725    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:57.727865 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.727769    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mz66v\" (UniqueName: \"kubernetes.io/projected/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-kube-api-access-mz66v\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:57.727865 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:57.727785    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:33:58.072347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:58.072309    5704 generic.go:356] "Generic (PLEG): container finished" podID="e44e6675-761e-427f-b3b5-4ccbd2bb0dbc" containerID="161040bcd6004fff71dfd0ed14f9679328d80fbc65bf3daa5cfd9459b75b0972" exitCode=0
Oct 30 19:33:58.072528 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:58.072362    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"
Oct 30 19:33:58.072528 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:58.072391    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc" event={"ID":"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc","Type":"ContainerDied","Data":"161040bcd6004fff71dfd0ed14f9679328d80fbc65bf3daa5cfd9459b75b0972"}
Oct 30 19:33:58.072528 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:58.072448    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc" event={"ID":"e44e6675-761e-427f-b3b5-4ccbd2bb0dbc","Type":"ContainerDied","Data":"d45c4f68fcacd245073acd604a9d3442a34e0854212a85f20f639decc76c728f"}
Oct 30 19:33:58.072528 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:58.072470    5704 scope.go:122] "RemoveContainer" containerID="161040bcd6004fff71dfd0ed14f9679328d80fbc65bf3daa5cfd9459b75b0972"
Oct 30 19:33:58.082196 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:58.082176    5704 scope.go:122] "RemoveContainer" containerID="161040bcd6004fff71dfd0ed14f9679328d80fbc65bf3daa5cfd9459b75b0972"
Oct 30 19:33:58.082438 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:33:58.082417    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"161040bcd6004fff71dfd0ed14f9679328d80fbc65bf3daa5cfd9459b75b0972\": container with ID starting with 161040bcd6004fff71dfd0ed14f9679328d80fbc65bf3daa5cfd9459b75b0972 not found: ID does not exist" containerID="161040bcd6004fff71dfd0ed14f9679328d80fbc65bf3daa5cfd9459b75b0972"
Oct 30 19:33:58.082490 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:58.082445    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"161040bcd6004fff71dfd0ed14f9679328d80fbc65bf3daa5cfd9459b75b0972"} err="failed to get container status \"161040bcd6004fff71dfd0ed14f9679328d80fbc65bf3daa5cfd9459b75b0972\": rpc error: code = NotFound desc = could not find container \"161040bcd6004fff71dfd0ed14f9679328d80fbc65bf3daa5cfd9459b75b0972\": container with ID starting with 161040bcd6004fff71dfd0ed14f9679328d80fbc65bf3daa5cfd9459b75b0972 not found: ID does not exist"
Oct 30 19:33:58.227643 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:58.227610    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"]
Oct 30 19:33:58.273472 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:58.273442    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-a-441836-865c866495-czznc"]
Oct 30 19:33:58.854404 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:58.854355    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:59.110124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:59.110021    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:59.179620 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:59.179588    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e44e6675-761e-427f-b3b5-4ccbd2bb0dbc" path="/var/lib/kubelet/pods/e44e6675-761e-427f-b3b5-4ccbd2bb0dbc/volumes"
Oct 30 19:33:59.307114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:59.307079    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:33:59.792982 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:33:59.792950    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:00.705082 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:00.705022    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:00.937065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:00.937023    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:03.382981 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:34:03.382949    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:34:04.001501 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:04.000161    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:06.427175 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:06.427135    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:06.610420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:06.610387    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:07.125089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.125051    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"]
Oct 30 19:34:07.125628 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.125611    5704 cpu_manager.go:414] "containerMap: removing container" podUID="020fbb14-fd8e-4941-9c3f-91309c71ce47" containerName="payload-pre-processing"
Oct 30 19:34:07.125628 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.125627    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="020fbb14-fd8e-4941-9c3f-91309c71ce47" containerName="payload-pre-processing"
Oct 30 19:34:07.125803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.125652    5704 cpu_manager.go:414] "containerMap: removing container" podUID="5357c9cd-0b77-44e6-930a-2b1f26884f9a" containerName="maas-api"
Oct 30 19:34:07.125803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.125657    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5357c9cd-0b77-44e6-930a-2b1f26884f9a" containerName="maas-api"
Oct 30 19:34:07.125803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.125667    5704 cpu_manager.go:414] "containerMap: removing container" podUID="82681f31-bc24-4592-8f1f-5ea5b852e40a" containerName="payload-processing"
Oct 30 19:34:07.125803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.125671    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="82681f31-bc24-4592-8f1f-5ea5b852e40a" containerName="payload-processing"
Oct 30 19:34:07.125803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.125684    5704 cpu_manager.go:414] "containerMap: removing container" podUID="e44e6675-761e-427f-b3b5-4ccbd2bb0dbc" containerName="maas-api"
Oct 30 19:34:07.125803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.125689    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e44e6675-761e-427f-b3b5-4ccbd2bb0dbc" containerName="maas-api"
Oct 30 19:34:07.125803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.125778    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="5357c9cd-0b77-44e6-930a-2b1f26884f9a" containerName="maas-api"
Oct 30 19:34:07.125803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.125787    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="82681f31-bc24-4592-8f1f-5ea5b852e40a" containerName="payload-processing"
Oct 30 19:34:07.125803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.125797    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="e44e6675-761e-427f-b3b5-4ccbd2bb0dbc" containerName="maas-api"
Oct 30 19:34:07.125803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.125808    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="020fbb14-fd8e-4941-9c3f-91309c71ce47" containerName="payload-pre-processing"
Oct 30 19:34:07.129168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.129148    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.231156 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.231125    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"]
Oct 30 19:34:07.231585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.231563    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.231645 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.231617    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ffd74\" (UniqueName: \"kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-kube-api-access-ffd74\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.231760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.231745    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.231804 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.231787    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.231885 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.231871    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.231929 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.231897    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.231969 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.231930    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.232005 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.231985    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.232063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.232021    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.333022 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.332989    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.333022 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.333028    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.333303 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.333083    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.333303 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.333153    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.333303 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.333222    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ffd74\" (UniqueName: \"kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-kube-api-access-ffd74\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.333471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.333353    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.333471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.333394    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.333471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.333454    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.333638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.333547    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.333638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.333606    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.333761 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.333646    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.333761 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.333672    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.333928 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.333908    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.334113 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.334085    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.335771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.335748    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.335956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.335940    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.391370 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.391308    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.391561 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.391538    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ffd74\" (UniqueName: \"kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-kube-api-access-ffd74\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.443192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.443163    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:07.874848 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.874816    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"]
Oct 30 19:34:07.880154 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:34:07.880118    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1036dd5_2b14_45fb_a368_b3ef1fccc7c6.slice/crio-449e49acaa7c36468f9d7f17c5b73776084c2f1a3ce2f8fccc1f5c3e6793bb3f WatchSource:0}: Error finding container 449e49acaa7c36468f9d7f17c5b73776084c2f1a3ce2f8fccc1f5c3e6793bb3f: Status 404 returned error can't find the container with id 449e49acaa7c36468f9d7f17c5b73776084c2f1a3ce2f8fccc1f5c3e6793bb3f
Oct 30 19:34:07.882695 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.882663    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:34:07.882776 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.882729    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:34:07.882776 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:07.882757    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:34:08.136236 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:08.136158    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" event={"ID":"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6","Type":"ContainerStarted","Data":"c8c0d1861cd31efc44d3db5377e04690855fba0743ab72f9818a7468330de773"}
Oct 30 19:34:08.136236 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:08.136205    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" event={"ID":"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6","Type":"ContainerStarted","Data":"449e49acaa7c36468f9d7f17c5b73776084c2f1a3ce2f8fccc1f5c3e6793bb3f"}
Oct 30 19:34:08.443557 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:08.443472    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:08.444893 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:08.444866    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 19:34:08.445017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:08.444920    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" podUID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 19:34:08.575620 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:08.575593    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:09.444581 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:09.444542    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 19:34:09.445122 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:09.444608    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" podUID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 19:34:10.204273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:10.204237    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:10.444415 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:10.444380    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 19:34:10.444592 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:10.444434    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" podUID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 19:34:11.045704 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:11.045663    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:11.444439 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:11.444355    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 19:34:11.444439 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:11.444408    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" podUID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 19:34:12.444386 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:12.444344    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 19:34:12.444912 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:12.444397    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" podUID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 19:34:13.199972 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:13.199923    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:13.342058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:13.337514    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" podStartSLOduration=6.337494599 podStartE2EDuration="6.337494599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:34:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:34:08.295793513 +0000 UTC m=+2821.241961416" watchObservedRunningTime="2026-10-30 19:34:13.337494599 +0000 UTC m=+2826.283662502"
Oct 30 19:34:13.344542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:13.344502    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:34:13.444430 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:13.444384    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 19:34:13.444867 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:13.444463    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" podUID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 19:34:14.443597 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:14.443566    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused" start-of-body=
Oct 30 19:34:14.443804 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:14.443630    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" podUID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.138:15021/healthz/ready\": dial tcp 10.134.0.138:15021: connect: connection refused"
Oct 30 19:34:15.719522 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:15.719489    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:15.719994 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:15.719970    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:15.720961 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:15.720936    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:34:16.788595 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:16.788556    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:18.229779 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:18.229735    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:21.302760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:21.302731    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:22.446940 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:22.446904    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:23.392145 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:23.392114    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:25.232061 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:25.232003    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:27.771465 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:27.771427    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"]
Oct 30 19:34:27.771982 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:27.771764    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerName="istio-proxy" containerID="cri-o://c545e76bbf9bd9343fdcede8ae25e25d9b7ac1734f8a8e7e38125cbf3a7f10ac" gracePeriod=30
Oct 30 19:34:29.407561 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:29.407525    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:29.578247 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:29.578210    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:31.986954 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:31.986918    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:33.046787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.046732    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 19:34:33.206983 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.206952    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-socket\") pod \"cc15c72d-8800-49fb-b221-6490dd1dfb45\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") "
Oct 30 19:34:33.207339 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.207313    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-socket" pod "cc15c72d-8800-49fb-b221-6490dd1dfb45" (UID: "cc15c72d-8800-49fb-b221-6490dd1dfb45"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:33.207472 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.207399    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-token\" (UniqueName: \"kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-token\") pod \"cc15c72d-8800-49fb-b221-6490dd1dfb45\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") "
Oct 30 19:34:33.207756 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.207735    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-podinfo\") pod \"cc15c72d-8800-49fb-b221-6490dd1dfb45\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") "
Oct 30 19:34:33.207838 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.207820    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-certs\") pod \"cc15c72d-8800-49fb-b221-6490dd1dfb45\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") "
Oct 30 19:34:33.207904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.207862    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-envoy\") pod \"cc15c72d-8800-49fb-b221-6490dd1dfb45\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") "
Oct 30 19:34:33.207951 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.207920    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-credential-socket\") pod \"cc15c72d-8800-49fb-b221-6490dd1dfb45\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") "
Oct 30 19:34:33.208617 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.208048    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-certs" pod "cc15c72d-8800-49fb-b221-6490dd1dfb45" (UID: "cc15c72d-8800-49fb-b221-6490dd1dfb45"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:33.208617 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.208270    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-credential-socket" pod "cc15c72d-8800-49fb-b221-6490dd1dfb45" (UID: "cc15c72d-8800-49fb-b221-6490dd1dfb45"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:33.209301 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.208889    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cc15c72d-8800-49fb-b221-6490dd1dfb45-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc15c72d-8800-49fb-b221-6490dd1dfb45-istiod-ca-cert\") pod \"cc15c72d-8800-49fb-b221-6490dd1dfb45\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") "
Oct 30 19:34:33.209558 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.209259    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc15c72d-8800-49fb-b221-6490dd1dfb45-istiod-ca-cert" pod "cc15c72d-8800-49fb-b221-6490dd1dfb45" (UID: "cc15c72d-8800-49fb-b221-6490dd1dfb45"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:34:33.210049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.209988    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-data" pod "cc15c72d-8800-49fb-b221-6490dd1dfb45" (UID: "cc15c72d-8800-49fb-b221-6490dd1dfb45"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:33.210203 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.210184    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-data\") pod \"cc15c72d-8800-49fb-b221-6490dd1dfb45\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") "
Oct 30 19:34:33.210373 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.210350    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-kube-api-access-5zmt2\" (UniqueName: \"kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-kube-api-access-5zmt2\") pod \"cc15c72d-8800-49fb-b221-6490dd1dfb45\" (UID: \"cc15c72d-8800-49fb-b221-6490dd1dfb45\") "
Oct 30 19:34:33.211072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.211010    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:33.211168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.211094    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:33.211168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.211109    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:33.211168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.211122    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cc15c72d-8800-49fb-b221-6490dd1dfb45-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:33.211168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.211133    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:33.211730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.211704    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-token" pod "cc15c72d-8800-49fb-b221-6490dd1dfb45" (UID: "cc15c72d-8800-49fb-b221-6490dd1dfb45"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:33.219613 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.219570    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-envoy" pod "cc15c72d-8800-49fb-b221-6490dd1dfb45" (UID: "cc15c72d-8800-49fb-b221-6490dd1dfb45"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:34:33.219873 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.219838    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-podinfo" pod "cc15c72d-8800-49fb-b221-6490dd1dfb45" (UID: "cc15c72d-8800-49fb-b221-6490dd1dfb45"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:34:33.226082 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.226047    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-kube-api-access-5zmt2" pod "cc15c72d-8800-49fb-b221-6490dd1dfb45" (UID: "cc15c72d-8800-49fb-b221-6490dd1dfb45"). InnerVolumeSpecName "kube-api-access-5zmt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:34:33.308641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.308552    5704 generic.go:356] "Generic (PLEG): container finished" podID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerID="c545e76bbf9bd9343fdcede8ae25e25d9b7ac1734f8a8e7e38125cbf3a7f10ac" exitCode=0
Oct 30 19:34:33.308641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.308597    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"
Oct 30 19:34:33.308837 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.308634    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" event={"ID":"cc15c72d-8800-49fb-b221-6490dd1dfb45","Type":"ContainerDied","Data":"c545e76bbf9bd9343fdcede8ae25e25d9b7ac1734f8a8e7e38125cbf3a7f10ac"}
Oct 30 19:34:33.308837 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.308688    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk" event={"ID":"cc15c72d-8800-49fb-b221-6490dd1dfb45","Type":"ContainerDied","Data":"4dff26fbf82dc62015a9b21437d45b8d8b47959c65124603de108937e8a5ec0d"}
Oct 30 19:34:33.308837 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.308712    5704 scope.go:122] "RemoveContainer" containerID="c545e76bbf9bd9343fdcede8ae25e25d9b7ac1734f8a8e7e38125cbf3a7f10ac"
Oct 30 19:34:33.312104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.312078    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:33.312220 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.312105    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:33.312220 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.312121    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cc15c72d-8800-49fb-b221-6490dd1dfb45-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:33.312220 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.312135    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zmt2\" (UniqueName: \"kubernetes.io/projected/cc15c72d-8800-49fb-b221-6490dd1dfb45-kube-api-access-5zmt2\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:34:33.318669 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.318648    5704 scope.go:122] "RemoveContainer" containerID="c545e76bbf9bd9343fdcede8ae25e25d9b7ac1734f8a8e7e38125cbf3a7f10ac"
Oct 30 19:34:33.318954 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:34:33.318936    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c545e76bbf9bd9343fdcede8ae25e25d9b7ac1734f8a8e7e38125cbf3a7f10ac\": container with ID starting with c545e76bbf9bd9343fdcede8ae25e25d9b7ac1734f8a8e7e38125cbf3a7f10ac not found: ID does not exist" containerID="c545e76bbf9bd9343fdcede8ae25e25d9b7ac1734f8a8e7e38125cbf3a7f10ac"
Oct 30 19:34:33.319016 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.318963    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c545e76bbf9bd9343fdcede8ae25e25d9b7ac1734f8a8e7e38125cbf3a7f10ac"} err="failed to get container status \"c545e76bbf9bd9343fdcede8ae25e25d9b7ac1734f8a8e7e38125cbf3a7f10ac\": rpc error: code = NotFound desc = could not find container \"c545e76bbf9bd9343fdcede8ae25e25d9b7ac1734f8a8e7e38125cbf3a7f10ac\": container with ID starting with c545e76bbf9bd9343fdcede8ae25e25d9b7ac1734f8a8e7e38125cbf3a7f10ac not found: ID does not exist"
Oct 30 19:34:33.390727 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:34:33.390688    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:34:33.470850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.470810    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"]
Oct 30 19:34:33.515977 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.515946    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8b64f9ff9-fd8nk"]
Oct 30 19:34:33.663822 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.663731    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:33.898963 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:33.898924    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:35.179248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:35.179209    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" path="/var/lib/kubelet/pods/cc15c72d-8800-49fb-b221-6490dd1dfb45/volumes"
Oct 30 19:34:36.597055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:36.597005    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:41.162937 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:41.162902    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:45.198828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:45.198793    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:46.654901 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:46.654867    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:46.886504 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:46.886462    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:47.069700 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:47.069658    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:47.657918 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:47.657881    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:51.810091 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:51.802356    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:54.311897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:54.311863    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:54.524742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:54.524709    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:54.702910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:54.702822    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:56.856585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:56.856550    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:34:59.159730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:34:59.159700    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:35:01.474389 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:01.474353    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:03.383709 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:35:03.383674    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:35:05.505510 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:05.505466    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:07.000785 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:07.000751    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:07.283274 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:07.283111    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:07.459762 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:07.459713    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:07.888908 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:07.888872    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:10.538312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:10.538279    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:13.107288 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.107253    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"]
Oct 30 19:35:13.107685 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.107455    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84" podUID="51d91680-b377-4411-83ed-46b4ebb90ea9" containerName="maas-api" containerID="cri-o://db30a8112b76d69dbc12785b8e1a35e06e9598310fb9409defe4d5f0c4f40684" gracePeriod=30
Oct 30 19:35:13.239818 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.239765    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:13.383718 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.383693    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:35:13.539644 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.539612    5704 generic.go:356] "Generic (PLEG): container finished" podID="51d91680-b377-4411-83ed-46b4ebb90ea9" containerID="db30a8112b76d69dbc12785b8e1a35e06e9598310fb9409defe4d5f0c4f40684" exitCode=0
Oct 30 19:35:13.539852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.539668    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84" event={"ID":"51d91680-b377-4411-83ed-46b4ebb90ea9","Type":"ContainerDied","Data":"db30a8112b76d69dbc12785b8e1a35e06e9598310fb9409defe4d5f0c4f40684"}
Oct 30 19:35:13.539852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.539708    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84" event={"ID":"51d91680-b377-4411-83ed-46b4ebb90ea9","Type":"ContainerDied","Data":"8608dda3b98834154ad910dbfb4a6955ece40fd1e6759fb7d87034f7398776d3"}
Oct 30 19:35:13.539852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.539728    5704 scope.go:122] "RemoveContainer" containerID="db30a8112b76d69dbc12785b8e1a35e06e9598310fb9409defe4d5f0c4f40684"
Oct 30 19:35:13.539852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.539671    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"
Oct 30 19:35:13.540297 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.540271    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"]
Oct 30 19:35:13.540499 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.540474    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz" podUID="516e0cbe-fd75-43ba-b635-c3427db43103" containerName="payload-processing" containerID="cri-o://82756fbdbc15dc7b313c9ce06866cd3107ba7af18406dea6097821454d43baea" gracePeriod=30
Oct 30 19:35:13.550576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.550552    5704 scope.go:122] "RemoveContainer" containerID="db30a8112b76d69dbc12785b8e1a35e06e9598310fb9409defe4d5f0c4f40684"
Oct 30 19:35:13.550835 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:35:13.550817    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db30a8112b76d69dbc12785b8e1a35e06e9598310fb9409defe4d5f0c4f40684\": container with ID starting with db30a8112b76d69dbc12785b8e1a35e06e9598310fb9409defe4d5f0c4f40684 not found: ID does not exist" containerID="db30a8112b76d69dbc12785b8e1a35e06e9598310fb9409defe4d5f0c4f40684"
Oct 30 19:35:13.550898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.550848    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db30a8112b76d69dbc12785b8e1a35e06e9598310fb9409defe4d5f0c4f40684"} err="failed to get container status \"db30a8112b76d69dbc12785b8e1a35e06e9598310fb9409defe4d5f0c4f40684\": rpc error: code = NotFound desc = could not find container \"db30a8112b76d69dbc12785b8e1a35e06e9598310fb9409defe4d5f0c4f40684\": container with ID starting with db30a8112b76d69dbc12785b8e1a35e06e9598310fb9409defe4d5f0c4f40684 not found: ID does not exist"
Oct 30 19:35:13.579237 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.579214    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/51d91680-b377-4411-83ed-46b4ebb90ea9-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51d91680-b377-4411-83ed-46b4ebb90ea9-openshift-service-ca\") pod \"51d91680-b377-4411-83ed-46b4ebb90ea9\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") "
Oct 30 19:35:13.579361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.579350    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-maas-api-tls\") pod \"51d91680-b377-4411-83ed-46b4ebb90ea9\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") "
Oct 30 19:35:13.579415 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.579375    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-metrics-certs\" (UniqueName: \"kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-metrics-certs\") pod \"51d91680-b377-4411-83ed-46b4ebb90ea9\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") "
Oct 30 19:35:13.579415 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.579402    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/51d91680-b377-4411-83ed-46b4ebb90ea9-kube-api-access-qhhm6\" (UniqueName: \"kubernetes.io/projected/51d91680-b377-4411-83ed-46b4ebb90ea9-kube-api-access-qhhm6\") pod \"51d91680-b377-4411-83ed-46b4ebb90ea9\" (UID: \"51d91680-b377-4411-83ed-46b4ebb90ea9\") "
Oct 30 19:35:13.579534 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.579509    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51d91680-b377-4411-83ed-46b4ebb90ea9-openshift-service-ca" pod "51d91680-b377-4411-83ed-46b4ebb90ea9" (UID: "51d91680-b377-4411-83ed-46b4ebb90ea9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:13.579850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.579828    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51d91680-b377-4411-83ed-46b4ebb90ea9-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:13.581405 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.581377    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-maas-api-tls" pod "51d91680-b377-4411-83ed-46b4ebb90ea9" (UID: "51d91680-b377-4411-83ed-46b4ebb90ea9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:13.581473 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.581420    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51d91680-b377-4411-83ed-46b4ebb90ea9-kube-api-access-qhhm6" pod "51d91680-b377-4411-83ed-46b4ebb90ea9" (UID: "51d91680-b377-4411-83ed-46b4ebb90ea9"). InnerVolumeSpecName "kube-api-access-qhhm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:13.581473 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.581446    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-metrics-certs" pod "51d91680-b377-4411-83ed-46b4ebb90ea9" (UID: "51d91680-b377-4411-83ed-46b4ebb90ea9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:35:13.592460 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.592432    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"]
Oct 30 19:35:13.592677 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.592657    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd" podUID="d50a816f-dd92-4f8a-ae64-8b4ccce066de" containerName="payload-pre-processing" containerID="cri-o://80321943d0932e3201a90cacba5c0f7bc473d6ccb34e465964454e30faf281f0" gracePeriod=30
Oct 30 19:35:13.680708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.680647    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:13.680708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.680675    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/51d91680-b377-4411-83ed-46b4ebb90ea9-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:13.680708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.680684    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhhm6\" (UniqueName: \"kubernetes.io/projected/51d91680-b377-4411-83ed-46b4ebb90ea9-kube-api-access-qhhm6\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:13.729850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.729823    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:13.849351 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.849330    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"
Oct 30 19:35:13.878840 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.878819    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"
Oct 30 19:35:13.885071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.885026    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/516e0cbe-fd75-43ba-b635-c3427db43103-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/516e0cbe-fd75-43ba-b635-c3427db43103-plugins-config-volume\") pod \"516e0cbe-fd75-43ba-b635-c3427db43103\" (UID: \"516e0cbe-fd75-43ba-b635-c3427db43103\") "
Oct 30 19:35:13.885178 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.885120    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d50a816f-dd92-4f8a-ae64-8b4ccce066de-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d50a816f-dd92-4f8a-ae64-8b4ccce066de-plugins-config-volume\") pod \"d50a816f-dd92-4f8a-ae64-8b4ccce066de\" (UID: \"d50a816f-dd92-4f8a-ae64-8b4ccce066de\") "
Oct 30 19:35:13.885221 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.885189    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/516e0cbe-fd75-43ba-b635-c3427db43103-kube-api-access-4d79w\" (UniqueName: \"kubernetes.io/projected/516e0cbe-fd75-43ba-b635-c3427db43103-kube-api-access-4d79w\") pod \"516e0cbe-fd75-43ba-b635-c3427db43103\" (UID: \"516e0cbe-fd75-43ba-b635-c3427db43103\") "
Oct 30 19:35:13.885260 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.885229    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d50a816f-dd92-4f8a-ae64-8b4ccce066de-kube-api-access-hwv9s\" (UniqueName: \"kubernetes.io/projected/d50a816f-dd92-4f8a-ae64-8b4ccce066de-kube-api-access-hwv9s\") pod \"d50a816f-dd92-4f8a-ae64-8b4ccce066de\" (UID: \"d50a816f-dd92-4f8a-ae64-8b4ccce066de\") "
Oct 30 19:35:13.885330 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.885310    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/516e0cbe-fd75-43ba-b635-c3427db43103-plugins-config-volume" pod "516e0cbe-fd75-43ba-b635-c3427db43103" (UID: "516e0cbe-fd75-43ba-b635-c3427db43103"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:13.885543 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.885523    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/516e0cbe-fd75-43ba-b635-c3427db43103-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:13.885543 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.885518    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d50a816f-dd92-4f8a-ae64-8b4ccce066de-plugins-config-volume" pod "d50a816f-dd92-4f8a-ae64-8b4ccce066de" (UID: "d50a816f-dd92-4f8a-ae64-8b4ccce066de"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:35:13.887299 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.887278    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d50a816f-dd92-4f8a-ae64-8b4ccce066de-kube-api-access-hwv9s" pod "d50a816f-dd92-4f8a-ae64-8b4ccce066de" (UID: "d50a816f-dd92-4f8a-ae64-8b4ccce066de"). InnerVolumeSpecName "kube-api-access-hwv9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:13.887299 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.887290    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/516e0cbe-fd75-43ba-b635-c3427db43103-kube-api-access-4d79w" pod "516e0cbe-fd75-43ba-b635-c3427db43103" (UID: "516e0cbe-fd75-43ba-b635-c3427db43103"). InnerVolumeSpecName "kube-api-access-4d79w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:35:13.919216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.919175    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:13.953343 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.953290    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"]
Oct 30 19:35:13.957892 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.957867    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-2a5c13-5d8648b8f-5gp84"]
Oct 30 19:35:13.986379 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.986359    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d50a816f-dd92-4f8a-ae64-8b4ccce066de-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:13.986379 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.986381    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d79w\" (UniqueName: \"kubernetes.io/projected/516e0cbe-fd75-43ba-b635-c3427db43103-kube-api-access-4d79w\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:13.986508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:13.986391    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwv9s\" (UniqueName: \"kubernetes.io/projected/d50a816f-dd92-4f8a-ae64-8b4ccce066de-kube-api-access-hwv9s\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:35:14.385399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.385363    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:14.565238 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.565202    5704 generic.go:356] "Generic (PLEG): container finished" podID="d50a816f-dd92-4f8a-ae64-8b4ccce066de" containerID="80321943d0932e3201a90cacba5c0f7bc473d6ccb34e465964454e30faf281f0" exitCode=0
Oct 30 19:35:14.565408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.565249    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"
Oct 30 19:35:14.565408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.565289    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd" event={"ID":"d50a816f-dd92-4f8a-ae64-8b4ccce066de","Type":"ContainerDied","Data":"80321943d0932e3201a90cacba5c0f7bc473d6ccb34e465964454e30faf281f0"}
Oct 30 19:35:14.565408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.565341    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd" event={"ID":"d50a816f-dd92-4f8a-ae64-8b4ccce066de","Type":"ContainerDied","Data":"0616231d4822529de0a21419840fcb2aa72f82638e2bb86d4e2bc76b15c8965a"}
Oct 30 19:35:14.565408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.565376    5704 scope.go:122] "RemoveContainer" containerID="80321943d0932e3201a90cacba5c0f7bc473d6ccb34e465964454e30faf281f0"
Oct 30 19:35:14.566853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.566834    5704 generic.go:356] "Generic (PLEG): container finished" podID="516e0cbe-fd75-43ba-b635-c3427db43103" containerID="82756fbdbc15dc7b313c9ce06866cd3107ba7af18406dea6097821454d43baea" exitCode=0
Oct 30 19:35:14.566947 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.566877    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"
Oct 30 19:35:14.566947 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.566878    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz" event={"ID":"516e0cbe-fd75-43ba-b635-c3427db43103","Type":"ContainerDied","Data":"82756fbdbc15dc7b313c9ce06866cd3107ba7af18406dea6097821454d43baea"}
Oct 30 19:35:14.566947 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.566925    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz" event={"ID":"516e0cbe-fd75-43ba-b635-c3427db43103","Type":"ContainerDied","Data":"6a1d78a06888326891a64321132a71501f012baf379b79591a282517e9c7427b"}
Oct 30 19:35:14.575704 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.575682    5704 scope.go:122] "RemoveContainer" containerID="80321943d0932e3201a90cacba5c0f7bc473d6ccb34e465964454e30faf281f0"
Oct 30 19:35:14.575980 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:35:14.575958    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80321943d0932e3201a90cacba5c0f7bc473d6ccb34e465964454e30faf281f0\": container with ID starting with 80321943d0932e3201a90cacba5c0f7bc473d6ccb34e465964454e30faf281f0 not found: ID does not exist" containerID="80321943d0932e3201a90cacba5c0f7bc473d6ccb34e465964454e30faf281f0"
Oct 30 19:35:14.576089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.575991    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80321943d0932e3201a90cacba5c0f7bc473d6ccb34e465964454e30faf281f0"} err="failed to get container status \"80321943d0932e3201a90cacba5c0f7bc473d6ccb34e465964454e30faf281f0\": rpc error: code = NotFound desc = could not find container \"80321943d0932e3201a90cacba5c0f7bc473d6ccb34e465964454e30faf281f0\": container with ID starting with 80321943d0932e3201a90cacba5c0f7bc473d6ccb34e465964454e30faf281f0 not found: ID does not exist"
Oct 30 19:35:14.576089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.576017    5704 scope.go:122] "RemoveContainer" containerID="82756fbdbc15dc7b313c9ce06866cd3107ba7af18406dea6097821454d43baea"
Oct 30 19:35:14.584583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.584567    5704 scope.go:122] "RemoveContainer" containerID="82756fbdbc15dc7b313c9ce06866cd3107ba7af18406dea6097821454d43baea"
Oct 30 19:35:14.584803 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:35:14.584785    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82756fbdbc15dc7b313c9ce06866cd3107ba7af18406dea6097821454d43baea\": container with ID starting with 82756fbdbc15dc7b313c9ce06866cd3107ba7af18406dea6097821454d43baea not found: ID does not exist" containerID="82756fbdbc15dc7b313c9ce06866cd3107ba7af18406dea6097821454d43baea"
Oct 30 19:35:14.584878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.584809    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82756fbdbc15dc7b313c9ce06866cd3107ba7af18406dea6097821454d43baea"} err="failed to get container status \"82756fbdbc15dc7b313c9ce06866cd3107ba7af18406dea6097821454d43baea\": rpc error: code = NotFound desc = could not find container \"82756fbdbc15dc7b313c9ce06866cd3107ba7af18406dea6097821454d43baea\": container with ID starting with 82756fbdbc15dc7b313c9ce06866cd3107ba7af18406dea6097821454d43baea not found: ID does not exist"
Oct 30 19:35:14.621994 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.621962    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:14.718097 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.718019    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"]
Oct 30 19:35:14.722382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.722361    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-2a5c13-6b6fc64854-dtzvz"]
Oct 30 19:35:14.825590 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.825560    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"]
Oct 30 19:35:14.828939 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.828916    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-2a5c13-7c58c59c9f-ssfrd"]
Oct 30 19:35:14.954763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:14.954736    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:15.128311 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:15.128274    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:15.179702 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:15.179663    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="516e0cbe-fd75-43ba-b635-c3427db43103" path="/var/lib/kubelet/pods/516e0cbe-fd75-43ba-b635-c3427db43103/volumes"
Oct 30 19:35:15.180209 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:15.180188    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="51d91680-b377-4411-83ed-46b4ebb90ea9" path="/var/lib/kubelet/pods/51d91680-b377-4411-83ed-46b4ebb90ea9/volumes"
Oct 30 19:35:15.180623 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:15.180604    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d50a816f-dd92-4f8a-ae64-8b4ccce066de" path="/var/lib/kubelet/pods/d50a816f-dd92-4f8a-ae64-8b4ccce066de/volumes"
Oct 30 19:35:15.314325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:15.314280    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:15.492127 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:15.492008    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:16.076530 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:16.076501    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:19.910762 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:19.910461    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:23.214048 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:23.214001    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:23.402922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:23.402884    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:23.594706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:23.594672    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:24.311992 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:24.311957    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:24.533069 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:24.533020    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:26.904654 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:26.904625    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:35:29.442126 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:29.442072    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:32.880421 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:32.880380    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:33.343672 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:33.343605    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:33.395623 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:35:33.395588    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:35:34.428418 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.428388    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"]
Oct 30 19:35:34.428902 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.428887    5704 cpu_manager.go:414] "containerMap: removing container" podUID="516e0cbe-fd75-43ba-b635-c3427db43103" containerName="payload-processing"
Oct 30 19:35:34.428902 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.428902    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="516e0cbe-fd75-43ba-b635-c3427db43103" containerName="payload-processing"
Oct 30 19:35:34.428986 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.428919    5704 cpu_manager.go:414] "containerMap: removing container" podUID="51d91680-b377-4411-83ed-46b4ebb90ea9" containerName="maas-api"
Oct 30 19:35:34.428986 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.428924    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="51d91680-b377-4411-83ed-46b4ebb90ea9" containerName="maas-api"
Oct 30 19:35:34.428986 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.428943    5704 cpu_manager.go:414] "containerMap: removing container" podUID="d50a816f-dd92-4f8a-ae64-8b4ccce066de" containerName="payload-pre-processing"
Oct 30 19:35:34.428986 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.428948    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d50a816f-dd92-4f8a-ae64-8b4ccce066de" containerName="payload-pre-processing"
Oct 30 19:35:34.428986 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.428965    5704 cpu_manager.go:414] "containerMap: removing container" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerName="istio-proxy"
Oct 30 19:35:34.428986 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.428970    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerName="istio-proxy"
Oct 30 19:35:34.429219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.429056    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="cc15c72d-8800-49fb-b221-6490dd1dfb45" containerName="istio-proxy"
Oct 30 19:35:34.429219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.429066    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="516e0cbe-fd75-43ba-b635-c3427db43103" containerName="payload-processing"
Oct 30 19:35:34.429219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.429075    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="51d91680-b377-4411-83ed-46b4ebb90ea9" containerName="maas-api"
Oct 30 19:35:34.429219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.429083    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="d50a816f-dd92-4f8a-ae64-8b4ccce066de" containerName="payload-pre-processing"
Oct 30 19:35:34.433947 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.433924    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"
Oct 30 19:35:34.483225 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.483200    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-83ac32-dockercfg-tfr2j\""
Oct 30 19:35:34.528850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.528815    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-83ac32\""
Oct 30 19:35:34.532415 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.532388    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c4e2902-6a3a-4c9c-a973-b38329849c19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl\" (UID: \"0c4e2902-6a3a-4c9c-a973-b38329849c19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"
Oct 30 19:35:34.532552 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.532446    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvkw8\" (UniqueName: \"kubernetes.io/projected/0c4e2902-6a3a-4c9c-a973-b38329849c19-kube-api-access-gvkw8\") pod \"payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl\" (UID: \"0c4e2902-6a3a-4c9c-a973-b38329849c19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"
Oct 30 19:35:34.532552 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.532523    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"]
Oct 30 19:35:34.633472 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.633442    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c4e2902-6a3a-4c9c-a973-b38329849c19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl\" (UID: \"0c4e2902-6a3a-4c9c-a973-b38329849c19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"
Oct 30 19:35:34.633608 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.633490    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvkw8\" (UniqueName: \"kubernetes.io/projected/0c4e2902-6a3a-4c9c-a973-b38329849c19-kube-api-access-gvkw8\") pod \"payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl\" (UID: \"0c4e2902-6a3a-4c9c-a973-b38329849c19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"
Oct 30 19:35:34.634152 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.634133    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c4e2902-6a3a-4c9c-a973-b38329849c19-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl\" (UID: \"0c4e2902-6a3a-4c9c-a973-b38329849c19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"
Oct 30 19:35:34.691295 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.691234    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvkw8\" (UniqueName: \"kubernetes.io/projected/0c4e2902-6a3a-4c9c-a973-b38329849c19-kube-api-access-gvkw8\") pod \"payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl\" (UID: \"0c4e2902-6a3a-4c9c-a973-b38329849c19\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"
Oct 30 19:35:34.747161 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:34.747141    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"
Oct 30 19:35:35.118927 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:35.118896    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"]
Oct 30 19:35:35.126013 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:35:35.125985    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c4e2902_6a3a_4c9c_a973_b38329849c19.slice/crio-ba586e52a86d0df9f8189187b30bf6aaac4995b992561127fdec592ed85b30cf WatchSource:0}: Error finding container ba586e52a86d0df9f8189187b30bf6aaac4995b992561127fdec592ed85b30cf: Status 404 returned error can't find the container with id ba586e52a86d0df9f8189187b30bf6aaac4995b992561127fdec592ed85b30cf
Oct 30 19:35:35.735098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:35.735062    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl" event={"ID":"0c4e2902-6a3a-4c9c-a973-b38329849c19","Type":"ContainerStarted","Data":"24ca1a93dcba307f5e8cda53d5a3d22a4a75a23e6525f5cd73f94deb0f00766f"}
Oct 30 19:35:35.735500 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:35.735120    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl" event={"ID":"0c4e2902-6a3a-4c9c-a973-b38329849c19","Type":"ContainerStarted","Data":"ba586e52a86d0df9f8189187b30bf6aaac4995b992561127fdec592ed85b30cf"}
Oct 30 19:35:35.735500 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:35.735268    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"
Oct 30 19:35:35.855905 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:35.855848    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl" podStartSLOduration=1.855832213 podStartE2EDuration="1.855832213s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:35.854914159 +0000 UTC m=+2908.801082062" watchObservedRunningTime="2026-10-30 19:35:35.855832213 +0000 UTC m=+2908.802000120"
Oct 30 19:35:35.947584 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:35.947557    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:40.590118 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:40.590069    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:45.196471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:45.196395    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:46.742648 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:46.742617    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"
Oct 30 19:35:49.621280 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:49.621251    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:35:49.981542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:49.981420    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:52.121663 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.121629    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"]
Oct 30 19:35:52.125564 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.125546    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.220828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.220800    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.220980 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.220848    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.221023 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.220974    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.221081 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.221045    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8758529a-cf50-4842-9124-4a8ce8499bf4-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.221129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.221110    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.221166 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.221143    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55wnw\" (UniqueName: \"kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-kube-api-access-55wnw\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.221201 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.221190    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.221318 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.221298    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.221377 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.221363    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.322267 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.322231    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.322413 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.322337    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.322413 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.322364    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.322413 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.322404    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.322585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.322449    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8758529a-cf50-4842-9124-4a8ce8499bf4-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.322585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.322476    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.322585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.322500    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55wnw\" (UniqueName: \"kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-kube-api-access-55wnw\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.322585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.322567    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.322801 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.322612    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-certs\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.322801 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.322618    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.322801 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.322690    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-data\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.322801 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.322788    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.323016 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.322863    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-credential-socket\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.323143 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.323124    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8758529a-cf50-4842-9124-4a8ce8499bf4-istiod-ca-cert\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.324940 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.324911    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-envoy\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.325155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.325140    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-podinfo\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.329859 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.329836    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"]
Oct 30 19:35:52.379891 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.379833    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-token\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.380008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.379960    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55wnw\" (UniqueName: \"kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-kube-api-access-55wnw\") pod \"e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") " pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.438945 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.438917    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:52.868681 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.868575    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"]
Oct 30 19:35:52.874692 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:35:52.874665    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8758529a_cf50_4842_9124_4a8ce8499bf4.slice/crio-3bb393d92c95d765f3af52cb47f022823fee76320d4dc8e233ed23cebc9a9b58 WatchSource:0}: Error finding container 3bb393d92c95d765f3af52cb47f022823fee76320d4dc8e233ed23cebc9a9b58: Status 404 returned error can't find the container with id 3bb393d92c95d765f3af52cb47f022823fee76320d4dc8e233ed23cebc9a9b58
Oct 30 19:35:52.877002 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.876971    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:35:52.877123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.877076    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:35:52.877166 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:52.877121    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:35:53.851105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:53.851065    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m" event={"ID":"8758529a-cf50-4842-9124-4a8ce8499bf4","Type":"ContainerStarted","Data":"8536cb2f66ba1d447d36743af9ca5f6e0847373f5561eb0d0f97bafd91cebeea"}
Oct 30 19:35:53.851507 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:53.851129    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m" event={"ID":"8758529a-cf50-4842-9124-4a8ce8499bf4","Type":"ContainerStarted","Data":"3bb393d92c95d765f3af52cb47f022823fee76320d4dc8e233ed23cebc9a9b58"}
Oct 30 19:35:54.014016 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:54.013963    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m" podStartSLOduration=2.013947832 podStartE2EDuration="2.013947832s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:35:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:35:54.011364853 +0000 UTC m=+2926.957532757" watchObservedRunningTime="2026-10-30 19:35:54.013947832 +0000 UTC m=+2926.960115734"
Oct 30 19:35:54.444796 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:54.444742    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:35:54.446186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:54.446160    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 19:35:54.446300 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:54.446220    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m" podUID="8758529a-cf50-4842-9124-4a8ce8499bf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 19:35:55.444456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:55.444415    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 19:35:55.444872 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:55.444497    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m" podUID="8758529a-cf50-4842-9124-4a8ce8499bf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 19:35:56.238083 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:56.235293    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:35:56.444324 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:56.444290    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 19:35:56.444496 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:56.444338    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m" podUID="8758529a-cf50-4842-9124-4a8ce8499bf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 19:35:57.443445 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:57.443409    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 19:35:57.443728 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:57.443703    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m" podUID="8758529a-cf50-4842-9124-4a8ce8499bf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 19:35:58.444483 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:58.444446    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 19:35:58.445025 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:58.444500    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m" podUID="8758529a-cf50-4842-9124-4a8ce8499bf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 19:35:59.439603 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:59.439568    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused" start-of-body=
Oct 30 19:35:59.439777 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:35:59.439621    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m" podUID="8758529a-cf50-4842-9124-4a8ce8499bf4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.140:15021/healthz/ready\": dial tcp 10.134.0.140:15021: connect: connection refused"
Oct 30 19:36:00.443410 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:00.443379    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:36:00.443887 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:00.443695    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:36:00.444438 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:00.444421    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:36:01.014289 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:01.014251    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:03.382906 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:36:03.382873    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:36:06.005552 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:06.003935    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:10.862973 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:10.862941    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:12.165594 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:12.165551    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:17.731775 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:17.731733    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:22.416021 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:22.415962    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:27.136888 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.136851    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:27.277933 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.277900    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"]
Oct 30 19:36:27.283458 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.283436    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.386334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.386307    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"]
Oct 30 19:36:27.413798 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.413720    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-2301cb-openshift-default-dockercfg-r29tl\""
Oct 30 19:36:27.455611 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.455580    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.455759 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.455623    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.455759 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.455714    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.455854 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.455829    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.455892 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.455858    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.455949 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.455933    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.455993 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.455975    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.456060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.456022    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vzwr\" (UniqueName: \"kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-kube-api-access-9vzwr\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.456177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.456158    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557070 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557014    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557244 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557094    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557244 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557111    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557244 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557137    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557244 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557222    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557244 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557240    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557297    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557335    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557673 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557383    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vzwr\" (UniqueName: \"kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-kube-api-access-9vzwr\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557673 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557640    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557673 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557572    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557521    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557785    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.557938 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.557914    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.559762 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.559741    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.559855 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.559817    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.614820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.614798    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.615057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.615010    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vzwr\" (UniqueName: \"kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-kube-api-access-9vzwr\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.897116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.897084    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:27.930323 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:27.930294    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:28.111529 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:28.111494    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:28.326870 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:28.326842    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"]
Oct 30 19:36:28.332558 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:36:28.332532    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc46c3c63_cf65_47cd_addc_41b0db2f23ae.slice/crio-5c17e2a809c0f7ae8b9c4d1de117b0614776696996578dcb12f20ee6d8d6e576 WatchSource:0}: Error finding container 5c17e2a809c0f7ae8b9c4d1de117b0614776696996578dcb12f20ee6d8d6e576: Status 404 returned error can't find the container with id 5c17e2a809c0f7ae8b9c4d1de117b0614776696996578dcb12f20ee6d8d6e576
Oct 30 19:36:28.334800 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:28.334767    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:28.334897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:28.334831    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:28.334897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:28.334859    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:29.078890 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:29.078853    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj" event={"ID":"c46c3c63-cf65-47cd-addc-41b0db2f23ae","Type":"ContainerStarted","Data":"7fabc7ffefe0bae99ae62cd820f445b8e8d2e065060420d1abd444fe97f4fd86"}
Oct 30 19:36:29.079091 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:29.078916    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj" event={"ID":"c46c3c63-cf65-47cd-addc-41b0db2f23ae","Type":"ContainerStarted","Data":"5c17e2a809c0f7ae8b9c4d1de117b0614776696996578dcb12f20ee6d8d6e576"}
Oct 30 19:36:29.897728 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:29.897694    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:29.899530 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:29.899503    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 19:36:29.899654 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:29.899558    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj" podUID="c46c3c63-cf65-47cd-addc-41b0db2f23ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 19:36:30.898025 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:30.897992    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 19:36:30.898460 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:30.898051    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj" podUID="c46c3c63-cf65-47cd-addc-41b0db2f23ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 19:36:31.071227 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.071138    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj" podStartSLOduration=4.071118617 podStartE2EDuration="4.071118617s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:29.246348213 +0000 UTC m=+2962.192516118" watchObservedRunningTime="2026-10-30 19:36:31.071118617 +0000 UTC m=+2964.017286521"
Oct 30 19:36:31.073698 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.073676    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"]
Oct 30 19:36:31.090099 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.090071    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8" podUID="8c84578b-0f06-46c9-a103-9f01bd06e5e7" containerName="maas-api" containerID="cri-o://39d62b82fe92981b4cf4a460e37bd3f93bc82a11f71dcaebb3afcbdd146c6e62" gracePeriod=30
Oct 30 19:36:31.203422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.203343    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:31.497547 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.497506    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:36:31.515534 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.515505    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-metrics-certs\") pod \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") "
Oct 30 19:36:31.515679 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.515576    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-maas-api-tls\") pod \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") "
Oct 30 19:36:31.515739 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.515681    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8c84578b-0f06-46c9-a103-9f01bd06e5e7-kube-api-access-krh5z\" (UniqueName: \"kubernetes.io/projected/8c84578b-0f06-46c9-a103-9f01bd06e5e7-kube-api-access-krh5z\") pod \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") "
Oct 30 19:36:31.515796 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.515765    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8c84578b-0f06-46c9-a103-9f01bd06e5e7-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c84578b-0f06-46c9-a103-9f01bd06e5e7-openshift-service-ca\") pod \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\" (UID: \"8c84578b-0f06-46c9-a103-9f01bd06e5e7\") "
Oct 30 19:36:31.516768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.516736    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c84578b-0f06-46c9-a103-9f01bd06e5e7-openshift-service-ca" pod "8c84578b-0f06-46c9-a103-9f01bd06e5e7" (UID: "8c84578b-0f06-46c9-a103-9f01bd06e5e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:31.528059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.516941    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8c84578b-0f06-46c9-a103-9f01bd06e5e7-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:31.552239 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.542154    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-maas-api-tls" pod "8c84578b-0f06-46c9-a103-9f01bd06e5e7" (UID: "8c84578b-0f06-46c9-a103-9f01bd06e5e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:31.558576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.558531    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-metrics-certs" pod "8c84578b-0f06-46c9-a103-9f01bd06e5e7" (UID: "8c84578b-0f06-46c9-a103-9f01bd06e5e7"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:36:31.560647 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.560620    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c84578b-0f06-46c9-a103-9f01bd06e5e7-kube-api-access-krh5z" pod "8c84578b-0f06-46c9-a103-9f01bd06e5e7" (UID: "8c84578b-0f06-46c9-a103-9f01bd06e5e7"). InnerVolumeSpecName "kube-api-access-krh5z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:31.597777 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.597750    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"]
Oct 30 19:36:31.598071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.598028    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b" podUID="769034b9-5ad4-46e5-be15-b79d307aa077" containerName="payload-pre-processing" containerID="cri-o://200d0de2d52a7a68f62131bc88836cd8d9c270a77e564d96c0df897855815627" gracePeriod=30
Oct 30 19:36:31.618796 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.618765    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:31.618922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.618799    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8c84578b-0f06-46c9-a103-9f01bd06e5e7-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:31.618922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.618816    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-krh5z\" (UniqueName: \"kubernetes.io/projected/8c84578b-0f06-46c9-a103-9f01bd06e5e7-kube-api-access-krh5z\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:31.708240 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.708207    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:31.782470 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.782397    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:36:31.896891 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.896864    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"
Oct 30 19:36:31.897364 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.897340    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 19:36:31.897461 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.897384    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj" podUID="c46c3c63-cf65-47cd-addc-41b0db2f23ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 19:36:31.944221 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.943884    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:31.946548 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.946522    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/769034b9-5ad4-46e5-be15-b79d307aa077-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/769034b9-5ad4-46e5-be15-b79d307aa077-plugins-config-volume\") pod \"769034b9-5ad4-46e5-be15-b79d307aa077\" (UID: \"769034b9-5ad4-46e5-be15-b79d307aa077\") "
Oct 30 19:36:31.946668 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.946630    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/769034b9-5ad4-46e5-be15-b79d307aa077-kube-api-access-pxc57\" (UniqueName: \"kubernetes.io/projected/769034b9-5ad4-46e5-be15-b79d307aa077-kube-api-access-pxc57\") pod \"769034b9-5ad4-46e5-be15-b79d307aa077\" (UID: \"769034b9-5ad4-46e5-be15-b79d307aa077\") "
Oct 30 19:36:31.946896 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.946874    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/769034b9-5ad4-46e5-be15-b79d307aa077-plugins-config-volume" pod "769034b9-5ad4-46e5-be15-b79d307aa077" (UID: "769034b9-5ad4-46e5-be15-b79d307aa077"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:36:31.947108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.947093    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/769034b9-5ad4-46e5-be15-b79d307aa077-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:31.948649 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:31.948630    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/769034b9-5ad4-46e5-be15-b79d307aa077-kube-api-access-pxc57" pod "769034b9-5ad4-46e5-be15-b79d307aa077" (UID: "769034b9-5ad4-46e5-be15-b79d307aa077"). InnerVolumeSpecName "kube-api-access-pxc57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:36:32.065107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.061074    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxc57\" (UniqueName: \"kubernetes.io/projected/769034b9-5ad4-46e5-be15-b79d307aa077-kube-api-access-pxc57\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:36:32.143949 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.143774    5704 generic.go:356] "Generic (PLEG): container finished" podID="8c84578b-0f06-46c9-a103-9f01bd06e5e7" containerID="39d62b82fe92981b4cf4a460e37bd3f93bc82a11f71dcaebb3afcbdd146c6e62" exitCode=0
Oct 30 19:36:32.143949 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.143875    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8" event={"ID":"8c84578b-0f06-46c9-a103-9f01bd06e5e7","Type":"ContainerDied","Data":"39d62b82fe92981b4cf4a460e37bd3f93bc82a11f71dcaebb3afcbdd146c6e62"}
Oct 30 19:36:32.146098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.144271    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8" event={"ID":"8c84578b-0f06-46c9-a103-9f01bd06e5e7","Type":"ContainerDied","Data":"80ee0356a09351dca9f0e16e6ec7f3ba9d4db3c7672e95675d344fc2128fedb5"}
Oct 30 19:36:32.146098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.144320    5704 scope.go:122] "RemoveContainer" containerID="39d62b82fe92981b4cf4a460e37bd3f93bc82a11f71dcaebb3afcbdd146c6e62"
Oct 30 19:36:32.146098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.144487    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"
Oct 30 19:36:32.158165 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.157916    5704 generic.go:356] "Generic (PLEG): container finished" podID="769034b9-5ad4-46e5-be15-b79d307aa077" containerID="200d0de2d52a7a68f62131bc88836cd8d9c270a77e564d96c0df897855815627" exitCode=0
Oct 30 19:36:32.162177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.158410    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b" event={"ID":"769034b9-5ad4-46e5-be15-b79d307aa077","Type":"ContainerDied","Data":"200d0de2d52a7a68f62131bc88836cd8d9c270a77e564d96c0df897855815627"}
Oct 30 19:36:32.162177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.158485    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b" event={"ID":"769034b9-5ad4-46e5-be15-b79d307aa077","Type":"ContainerDied","Data":"ebeba1ecfd650a18e5075bf572b058f893bfe182c1ab226fb69c4144afdff60d"}
Oct 30 19:36:32.162177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.158578    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"
Oct 30 19:36:32.175265 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.175249    5704 scope.go:122] "RemoveContainer" containerID="39d62b82fe92981b4cf4a460e37bd3f93bc82a11f71dcaebb3afcbdd146c6e62"
Oct 30 19:36:32.175553 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:36:32.175526    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39d62b82fe92981b4cf4a460e37bd3f93bc82a11f71dcaebb3afcbdd146c6e62\": container with ID starting with 39d62b82fe92981b4cf4a460e37bd3f93bc82a11f71dcaebb3afcbdd146c6e62 not found: ID does not exist" containerID="39d62b82fe92981b4cf4a460e37bd3f93bc82a11f71dcaebb3afcbdd146c6e62"
Oct 30 19:36:32.175658 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.175557    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39d62b82fe92981b4cf4a460e37bd3f93bc82a11f71dcaebb3afcbdd146c6e62"} err="failed to get container status \"39d62b82fe92981b4cf4a460e37bd3f93bc82a11f71dcaebb3afcbdd146c6e62\": rpc error: code = NotFound desc = could not find container \"39d62b82fe92981b4cf4a460e37bd3f93bc82a11f71dcaebb3afcbdd146c6e62\": container with ID starting with 39d62b82fe92981b4cf4a460e37bd3f93bc82a11f71dcaebb3afcbdd146c6e62 not found: ID does not exist"
Oct 30 19:36:32.175658 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.175576    5704 scope.go:122] "RemoveContainer" containerID="200d0de2d52a7a68f62131bc88836cd8d9c270a77e564d96c0df897855815627"
Oct 30 19:36:32.184407 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.184390    5704 scope.go:122] "RemoveContainer" containerID="200d0de2d52a7a68f62131bc88836cd8d9c270a77e564d96c0df897855815627"
Oct 30 19:36:32.184657 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:36:32.184639    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"200d0de2d52a7a68f62131bc88836cd8d9c270a77e564d96c0df897855815627\": container with ID starting with 200d0de2d52a7a68f62131bc88836cd8d9c270a77e564d96c0df897855815627 not found: ID does not exist" containerID="200d0de2d52a7a68f62131bc88836cd8d9c270a77e564d96c0df897855815627"
Oct 30 19:36:32.184709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.184664    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"200d0de2d52a7a68f62131bc88836cd8d9c270a77e564d96c0df897855815627"} err="failed to get container status \"200d0de2d52a7a68f62131bc88836cd8d9c270a77e564d96c0df897855815627\": rpc error: code = NotFound desc = could not find container \"200d0de2d52a7a68f62131bc88836cd8d9c270a77e564d96c0df897855815627\": container with ID starting with 200d0de2d52a7a68f62131bc88836cd8d9c270a77e564d96c0df897855815627 not found: ID does not exist"
Oct 30 19:36:32.303454 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.303424    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"]
Oct 30 19:36:32.309125 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.309102    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-f3b5af-d5f766d6b-wqpn8"]
Oct 30 19:36:32.409802 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.409733    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"]
Oct 30 19:36:32.414358 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.414332    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-f3b5af-84fff4ccc9-8hb4b"]
Oct 30 19:36:32.745462 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.745438    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:32.900059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.899387    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 19:36:32.900059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.899449    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj" podUID="c46c3c63-cf65-47cd-addc-41b0db2f23ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 19:36:32.995064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:32.986644    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:33.168112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:33.167135    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:33.214585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:33.214547    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="769034b9-5ad4-46e5-be15-b79d307aa077" path="/var/lib/kubelet/pods/769034b9-5ad4-46e5-be15-b79d307aa077/volumes"
Oct 30 19:36:33.215711 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:33.215688    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8c84578b-0f06-46c9-a103-9f01bd06e5e7" path="/var/lib/kubelet/pods/8c84578b-0f06-46c9-a103-9f01bd06e5e7/volumes"
Oct 30 19:36:33.487599 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:36:33.487509    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:36:33.692067 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:33.692016    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:33.898058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:33.897749    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 19:36:33.898058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:33.897809    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj" podUID="c46c3c63-cf65-47cd-addc-41b0db2f23ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 19:36:34.897534 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:34.897497    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused" start-of-body=
Oct 30 19:36:34.897927 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:34.897569    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj" podUID="c46c3c63-cf65-47cd-addc-41b0db2f23ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": dial tcp 10.134.0.141:15021: connect: connection refused"
Oct 30 19:36:36.938176 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:36.938140    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.141:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:36:36.938689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:36.938197    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj" podUID="c46c3c63-cf65-47cd-addc-41b0db2f23ae" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.141:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:36:37.066916 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:37.066885    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:37.067628 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:37.067603    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:37.068894 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:37.068870    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:36:38.188734 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:38.188694    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:41.077982 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:41.077939    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:41.311906 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:41.311868    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:41.529730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:41.529701    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:42.897110 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:42.897060    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:47.685633 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:47.685599    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:53.168468 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.168437    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:53.668224 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.668190    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"]
Oct 30 19:36:53.668765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.668751    5704 cpu_manager.go:414] "containerMap: removing container" podUID="8c84578b-0f06-46c9-a103-9f01bd06e5e7" containerName="maas-api"
Oct 30 19:36:53.668914 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.668767    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8c84578b-0f06-46c9-a103-9f01bd06e5e7" containerName="maas-api"
Oct 30 19:36:53.668914 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.668787    5704 cpu_manager.go:414] "containerMap: removing container" podUID="769034b9-5ad4-46e5-be15-b79d307aa077" containerName="payload-pre-processing"
Oct 30 19:36:53.668914 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.668796    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="769034b9-5ad4-46e5-be15-b79d307aa077" containerName="payload-pre-processing"
Oct 30 19:36:53.668914 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.668912    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="8c84578b-0f06-46c9-a103-9f01bd06e5e7" containerName="maas-api"
Oct 30 19:36:53.669202 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.668923    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="769034b9-5ad4-46e5-be15-b79d307aa077" containerName="payload-pre-processing"
Oct 30 19:36:53.673074 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.673055    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.762059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.761879    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-db06cf-openshift-default-dockercfg-f69wp\""
Oct 30 19:36:53.792490 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.792463    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:53.808222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.808193    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"]
Oct 30 19:36:53.853298 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.852439    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-certs\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.853298 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.852883    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istiod-ca-cert\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.853298 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.853117    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-socket\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.853620 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.853340    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-token\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.853620 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.853386    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-podinfo\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.859290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.853932    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-envoy\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.859290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.854362    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-data\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.859290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.854404    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-credential-socket\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.859290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.854649    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9wkt\" (UniqueName: \"kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-kube-api-access-j9wkt\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.956509 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.956431    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-envoy\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.956672 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.956513    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-data\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.956672 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.956634    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-credential-socket\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.956793 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.956686    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9wkt\" (UniqueName: \"kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-kube-api-access-j9wkt\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.956793 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.956773    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-certs\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.956928 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.956822    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istiod-ca-cert\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.956928 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.956878    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-socket\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.956928 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.956886    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-data\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.956928 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.956920    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-token\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.957197 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.956958    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-podinfo\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.957197 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.957177    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-certs\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.957318 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.957230    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-socket\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.957389 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.957336    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-credential-socket\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.957581 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.957556    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istiod-ca-cert\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.958792 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.958766    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-envoy\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.959438 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.959417    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-podinfo\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:53.978639 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:53.978611    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:54.015443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:54.015407    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-token\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:54.015443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:54.015428    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9wkt\" (UniqueName: \"kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-kube-api-access-j9wkt\") pod \"e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") " pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:54.202175 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:54.202139    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:54.286123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:54.286094    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:54.713413 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:54.713338    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"]
Oct 30 19:36:54.720189 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:36:54.720166    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f538c70_f8bf_4a0a_bcc0_6800a8b495ce.slice/crio-adb247c73ee57a80b5850ebcb620bb2f6c7b8ca77c0a851505ebf20191ebed15 WatchSource:0}: Error finding container adb247c73ee57a80b5850ebcb620bb2f6c7b8ca77c0a851505ebf20191ebed15: Status 404 returned error can't find the container with id adb247c73ee57a80b5850ebcb620bb2f6c7b8ca77c0a851505ebf20191ebed15
Oct 30 19:36:54.722508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:54.722474    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:54.722594 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:54.722543    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:54.722594 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:54.722574    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:55.347637 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:55.347604    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" event={"ID":"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce","Type":"ContainerStarted","Data":"62a43352f0135bbcbc52dbabfea0482e953e6ae252895e875f2c94edeed4b790"}
Oct 30 19:36:55.348025 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:55.347652    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" event={"ID":"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce","Type":"ContainerStarted","Data":"adb247c73ee57a80b5850ebcb620bb2f6c7b8ca77c0a851505ebf20191ebed15"}
Oct 30 19:36:55.508329 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:55.508283    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" podStartSLOduration=2.508267889 podStartE2EDuration="2.508267889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:55.50722795 +0000 UTC m=+2988.453395855" watchObservedRunningTime="2026-10-30 19:36:55.508267889 +0000 UTC m=+2988.454435792"
Oct 30 19:36:56.286377 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:56.286341    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:56.291581 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:56.291559    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:56.353720 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:56.353693    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:56.354434 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:56.354418    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:36:57.521916 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.521878    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"]
Oct 30 19:36:57.525980 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.525962    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.600501 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.600468    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.600680 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.600508    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.600759 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.600687    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.600759 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.600731    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.600852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.600777    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.600852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.600841    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9ltl\" (UniqueName: \"kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-kube-api-access-m9ltl\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.600937 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.600886    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.600937 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.600919    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.601048 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.600958    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.623175 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.623145    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"]
Oct 30 19:36:57.701913 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.701881    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.702116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.701925    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.702116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.701944    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.702116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.702027    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.702116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.702070    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.702296 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.702157    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.702296 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.702181    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.702296 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.702210    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.702296 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.702265    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9ltl\" (UniqueName: \"kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-kube-api-access-m9ltl\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.702486 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.702328    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-credential-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.702486 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.702397    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-socket\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.702595 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.702522    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-data\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.702595 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.702585    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-certs\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.703082 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.703057    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istiod-ca-cert\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.704538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.704518    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-envoy\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.704651 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.704632    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-podinfo\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.759536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.759503    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-token\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.759700 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.759560    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9ltl\" (UniqueName: \"kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-kube-api-access-m9ltl\") pod \"e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") " pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:57.840566 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:57.840483    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:58.260972 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:58.260940    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"]
Oct 30 19:36:58.263973 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:36:58.263947    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ec3ec9b_9c71_4b27_91ef_0bab48eafad6.slice/crio-3587451a8882cfc09236de27f9e598aa46f84d630c338ae95ce901c0f47c1164 WatchSource:0}: Error finding container 3587451a8882cfc09236de27f9e598aa46f84d630c338ae95ce901c0f47c1164: Status 404 returned error can't find the container with id 3587451a8882cfc09236de27f9e598aa46f84d630c338ae95ce901c0f47c1164
Oct 30 19:36:58.266528 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:58.266497    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:58.266600 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:58.266562    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:58.266600 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:58.266590    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:36:58.330370 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:58.330340    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:58.367666 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:58.367628    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5" event={"ID":"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6","Type":"ContainerStarted","Data":"3bcf2ffac38f0be5395c0663ed3917fa508f174ea1016d7145df6a81ba2ba4dc"}
Oct 30 19:36:58.367819 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:58.367697    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5" event={"ID":"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6","Type":"ContainerStarted","Data":"3587451a8882cfc09236de27f9e598aa46f84d630c338ae95ce901c0f47c1164"}
Oct 30 19:36:58.513443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:58.513328    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5" podStartSLOduration=1.5133085670000002 podStartE2EDuration="1.513308567s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:36:58.511862221 +0000 UTC m=+2991.458030123" watchObservedRunningTime="2026-10-30 19:36:58.513308567 +0000 UTC m=+2991.459476474"
Oct 30 19:36:58.597879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:58.597835    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:58.781056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:58.780909    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:58.843562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:58.843525    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:36:59.164385 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:59.164279    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:36:59.846905 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:59.846873    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 30 19:36:59.847341 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:36:59.846930    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5" podUID="3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 30 19:37:00.844521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:00.844485    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 30 19:37:00.844749 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:00.844578    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5" podUID="3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 30 19:37:01.562950 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:01.562912    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:01.747764 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:01.747735    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:01.865116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:01.857404    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 30 19:37:01.865116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:01.857474    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5" podUID="3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 30 19:37:02.110154 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.109391    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"]
Oct 30 19:37:02.148112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.147351    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"
Oct 30 19:37:02.170731 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.170705    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"]
Oct 30 19:37:02.213909 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.213879    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-db06cf\""
Oct 30 19:37:02.214075 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.214018    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-db06cf-dockercfg-bz4vq\""
Oct 30 19:37:02.347763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.347730    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7\" (UID: \"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"
Oct 30 19:37:02.347942 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.347793    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz5v5\" (UniqueName: \"kubernetes.io/projected/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-kube-api-access-rz5v5\") pod \"payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7\" (UID: \"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"
Oct 30 19:37:02.448985 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.448902    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7\" (UID: \"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"
Oct 30 19:37:02.449177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.449066    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz5v5\" (UniqueName: \"kubernetes.io/projected/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-kube-api-access-rz5v5\") pod \"payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7\" (UID: \"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"
Oct 30 19:37:02.449719 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.449698    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7\" (UID: \"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"
Oct 30 19:37:02.506831 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.506801    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz5v5\" (UniqueName: \"kubernetes.io/projected/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-kube-api-access-rz5v5\") pod \"payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7\" (UID: \"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"
Oct 30 19:37:02.777058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.777004    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"
Oct 30 19:37:02.844410 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.844371    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 30 19:37:02.844598 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:02.844448    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5" podUID="3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 30 19:37:03.176292 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:37:03.176258    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fdbaa8d_ad3f_4202_9ddc_e208538a14ac.slice/crio-d937a59edfed4cb3ebbf81ac178e871cf8b08ca088fbd6fa6cf523f2b45d6a36 WatchSource:0}: Error finding container d937a59edfed4cb3ebbf81ac178e871cf8b08ca088fbd6fa6cf523f2b45d6a36: Status 404 returned error can't find the container with id d937a59edfed4cb3ebbf81ac178e871cf8b08ca088fbd6fa6cf523f2b45d6a36
Oct 30 19:37:03.180735 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:03.180713    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"]
Oct 30 19:37:03.384268 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:37:03.384236    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:37:03.397937 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:03.397911    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7" event={"ID":"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac","Type":"ContainerStarted","Data":"3658ce1f7bddfff7b91b2347090369c6e81254b1a96d389bafe64d4a761631ad"}
Oct 30 19:37:03.398112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:03.397957    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7" event={"ID":"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac","Type":"ContainerStarted","Data":"d937a59edfed4cb3ebbf81ac178e871cf8b08ca088fbd6fa6cf523f2b45d6a36"}
Oct 30 19:37:03.398112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:03.398003    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"
Oct 30 19:37:03.513141 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:03.513089    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7" podStartSLOduration=1.513075199 podStartE2EDuration="1.513075199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:03.512543702 +0000 UTC m=+2996.458711605" watchObservedRunningTime="2026-10-30 19:37:03.513075199 +0000 UTC m=+2996.459243102"
Oct 30 19:37:03.842247 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:03.842149    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 30 19:37:03.842853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:03.842806    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5" podUID="3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 30 19:37:04.348062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:04.345244    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:04.841604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:04.841575    5704 patch_prober.go:29] interesting pod/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused" start-of-body=
Oct 30 19:37:04.841783 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:04.841638    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5" podUID="3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.143:15021/healthz/ready\": dial tcp 10.134.0.143:15021: connect: connection refused"
Oct 30 19:37:05.845317 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:05.845291    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:37:05.845731 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:05.845716    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:37:05.846325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:05.846301    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:37:09.165164 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:09.165125    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:09.595922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:09.595884    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:11.198748 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:11.198710    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:13.171206 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:13.171171    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:13.567501 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:13.567465    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:37:13.785806 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:13.785771    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:13.995360 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:13.995285    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"]
Oct 30 19:37:14.007583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.007556    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.064085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.064052    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"]
Oct 30 19:37:14.102639 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.102609    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-85f471-openshift-default-dockercfg-nrxnn\""
Oct 30 19:37:14.108510 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.108484    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-credential-socket\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.108644 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.108522    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-token\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.108644 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.108631    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istiod-ca-cert\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.108755 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.108660    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-socket\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.108755 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.108686    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-certs\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.108864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.108844    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-envoy\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.108924 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.108911    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6t49v\" (UniqueName: \"kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-kube-api-access-6t49v\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.108964 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.108957    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-podinfo\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.109093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.109074    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-data\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.125685 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.125656    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:14.221872 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.221829    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istiod-ca-cert\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.223304 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.223281    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-socket\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.223517 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.223499    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-certs\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.224316 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.224296    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-envoy\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.224535 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.224517    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6t49v\" (UniqueName: \"kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-kube-api-access-6t49v\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.225340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.225319    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-podinfo\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.225620 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.225601    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-data\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.225748 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.225733    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-credential-socket\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.225871 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.225857    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-token\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.227520 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.223901    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-socket\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.227725 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.224139    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-certs\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.227852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.223214    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istiod-ca-cert\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.227979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.226873    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-data\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.232148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.227107    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-credential-socket\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.271055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.264191    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-envoy\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.274656 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.274627    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-podinfo\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.287461 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.287439    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-token\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.287754 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.287731    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6t49v\" (UniqueName: \"kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-kube-api-access-6t49v\") pod \"e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") " pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.320952 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.320927    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:14.391892 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.391860    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:14.406114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.406092    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"
Oct 30 19:37:14.782269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.770551    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"]
Oct 30 19:37:14.788428 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.787605    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:14.788428 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.787698    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:14.788428 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.787739    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:37:14.821431 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:14.821294    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:15.497353 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:15.497302    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" event={"ID":"139cc6a4-643f-41ee-a8c2-eaf2177f4052","Type":"ContainerStarted","Data":"47071d4e5aea22e8e246f9949e8093c5068faf1195413c78c5d5cd85fa934b5e"}
Oct 30 19:37:15.497994 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:15.497968    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" event={"ID":"139cc6a4-643f-41ee-a8c2-eaf2177f4052","Type":"ContainerStarted","Data":"d1657696da89222dcfbc737ab91fdb165d94729eff3d4714c82bfc23c5d647d5"}
Oct 30 19:37:15.717541 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:15.717490    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" podStartSLOduration=2.717476888 podStartE2EDuration="2.717476888s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:15.716509142 +0000 UTC m=+3008.662677048" watchObservedRunningTime="2026-10-30 19:37:15.717476888 +0000 UTC m=+3008.663644788"
Oct 30 19:37:16.321489 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:16.321457    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:16.326817 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:16.326794    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:16.502753 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:16.502725    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:16.503712 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:16.503695    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:37:18.166093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:18.166054    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:18.749554 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:18.749500    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:18.937654 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:18.936024    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:19.122672 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:19.122560    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:22.769656 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:22.769615    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:23.001026 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.000993    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:23.409325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.409292    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"]
Oct 30 19:37:23.409493 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.409474    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl" podUID="0c4e2902-6a3a-4c9c-a973-b38329849c19" containerName="payload-pre-processing" containerID="cri-o://24ca1a93dcba307f5e8cda53d5a3d22a4a75a23e6525f5cd73f94deb0f00766f" gracePeriod=30
Oct 30 19:37:23.534186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.534151    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:23.546174 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.546141    5704 generic.go:356] "Generic (PLEG): container finished" podID="0c4e2902-6a3a-4c9c-a973-b38329849c19" containerID="24ca1a93dcba307f5e8cda53d5a3d22a4a75a23e6525f5cd73f94deb0f00766f" exitCode=0
Oct 30 19:37:23.546299 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.546210    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl" event={"ID":"0c4e2902-6a3a-4c9c-a973-b38329849c19","Type":"ContainerDied","Data":"24ca1a93dcba307f5e8cda53d5a3d22a4a75a23e6525f5cd73f94deb0f00766f"}
Oct 30 19:37:23.669827 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.669775    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"
Oct 30 19:37:23.695185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.695163    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0c4e2902-6a3a-4c9c-a973-b38329849c19-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c4e2902-6a3a-4c9c-a973-b38329849c19-plugins-config-volume\") pod \"0c4e2902-6a3a-4c9c-a973-b38329849c19\" (UID: \"0c4e2902-6a3a-4c9c-a973-b38329849c19\") "
Oct 30 19:37:23.695331 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.695320    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0c4e2902-6a3a-4c9c-a973-b38329849c19-kube-api-access-gvkw8\" (UniqueName: \"kubernetes.io/projected/0c4e2902-6a3a-4c9c-a973-b38329849c19-kube-api-access-gvkw8\") pod \"0c4e2902-6a3a-4c9c-a973-b38329849c19\" (UID: \"0c4e2902-6a3a-4c9c-a973-b38329849c19\") "
Oct 30 19:37:23.695500 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.695473    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c4e2902-6a3a-4c9c-a973-b38329849c19-plugins-config-volume" pod "0c4e2902-6a3a-4c9c-a973-b38329849c19" (UID: "0c4e2902-6a3a-4c9c-a973-b38329849c19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:37:23.695693 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.695679    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c4e2902-6a3a-4c9c-a973-b38329849c19-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:23.697315 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.697294    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c4e2902-6a3a-4c9c-a973-b38329849c19-kube-api-access-gvkw8" pod "0c4e2902-6a3a-4c9c-a973-b38329849c19" (UID: "0c4e2902-6a3a-4c9c-a973-b38329849c19"). InnerVolumeSpecName "kube-api-access-gvkw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:37:23.720773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.720748    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:23.796884 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:23.796855    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gvkw8\" (UniqueName: \"kubernetes.io/projected/0c4e2902-6a3a-4c9c-a973-b38329849c19-kube-api-access-gvkw8\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:37:24.553889 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:24.553860    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"
Oct 30 19:37:24.554130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:24.553911    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl" event={"ID":"0c4e2902-6a3a-4c9c-a973-b38329849c19","Type":"ContainerDied","Data":"ba586e52a86d0df9f8189187b30bf6aaac4995b992561127fdec592ed85b30cf"}
Oct 30 19:37:24.554130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:24.554010    5704 scope.go:122] "RemoveContainer" containerID="24ca1a93dcba307f5e8cda53d5a3d22a4a75a23e6525f5cd73f94deb0f00766f"
Oct 30 19:37:24.708312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:24.708283    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"]
Oct 30 19:37:24.754161 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:24.754132    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-83ac32-66495bf99-mcmpl"]
Oct 30 19:37:24.911514 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:24.911406    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:25.181394 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:25.181311    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0c4e2902-6a3a-4c9c-a973-b38329849c19" path="/var/lib/kubelet/pods/0c4e2902-6a3a-4c9c-a973-b38329849c19/volumes"
Oct 30 19:37:25.420196 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:25.420163    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:25.648355 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:25.648316    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:25.880829 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:25.880780    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:26.121086 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.121002    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:26.335065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.335006    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:26.510020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.509991    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"]
Oct 30 19:37:26.510893 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.510864    5704 cpu_manager.go:414] "containerMap: removing container" podUID="0c4e2902-6a3a-4c9c-a973-b38329849c19" containerName="payload-pre-processing"
Oct 30 19:37:26.510893 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.510895    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0c4e2902-6a3a-4c9c-a973-b38329849c19" containerName="payload-pre-processing"
Oct 30 19:37:26.514885 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.514827    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="0c4e2902-6a3a-4c9c-a973-b38329849c19" containerName="payload-pre-processing"
Oct 30 19:37:26.520647 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.520626    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"
Oct 30 19:37:26.569110 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.569080    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1daee16d-a4f6-4f87-b792-b3866450b2b2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59\" (UID: \"1daee16d-a4f6-4f87-b792-b3866450b2b2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"
Oct 30 19:37:26.569292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.569278    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srt2b\" (UniqueName: \"kubernetes.io/projected/1daee16d-a4f6-4f87-b792-b3866450b2b2-kube-api-access-srt2b\") pod \"payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59\" (UID: \"1daee16d-a4f6-4f87-b792-b3866450b2b2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"
Oct 30 19:37:26.574750 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.574728    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"]
Oct 30 19:37:26.581393 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.581368    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:26.617992 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.617966    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w0-2186d8\""
Oct 30 19:37:26.618168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.618138    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w0-2186d8-dockercfg-gtdj6\""
Oct 30 19:37:26.670959 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.670929    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srt2b\" (UniqueName: \"kubernetes.io/projected/1daee16d-a4f6-4f87-b792-b3866450b2b2-kube-api-access-srt2b\") pod \"payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59\" (UID: \"1daee16d-a4f6-4f87-b792-b3866450b2b2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"
Oct 30 19:37:26.671120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.671103    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1daee16d-a4f6-4f87-b792-b3866450b2b2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59\" (UID: \"1daee16d-a4f6-4f87-b792-b3866450b2b2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"
Oct 30 19:37:26.671808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.671791    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1daee16d-a4f6-4f87-b792-b3866450b2b2-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59\" (UID: \"1daee16d-a4f6-4f87-b792-b3866450b2b2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"
Oct 30 19:37:26.728414 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.728390    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srt2b\" (UniqueName: \"kubernetes.io/projected/1daee16d-a4f6-4f87-b792-b3866450b2b2-kube-api-access-srt2b\") pod \"payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59\" (UID: \"1daee16d-a4f6-4f87-b792-b3866450b2b2\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"
Oct 30 19:37:26.836538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:26.836459    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"
Oct 30 19:37:27.212888 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:27.212862    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"]
Oct 30 19:37:27.215436 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:37:27.215408    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1daee16d_a4f6_4f87_b792_b3866450b2b2.slice/crio-568c235d3eb781d0bcb5883e2080161b0bd790fe48a24b5f6477cf02dda6a82a WatchSource:0}: Error finding container 568c235d3eb781d0bcb5883e2080161b0bd790fe48a24b5f6477cf02dda6a82a: Status 404 returned error can't find the container with id 568c235d3eb781d0bcb5883e2080161b0bd790fe48a24b5f6477cf02dda6a82a
Oct 30 19:37:27.610123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:27.610081    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59" event={"ID":"1daee16d-a4f6-4f87-b792-b3866450b2b2","Type":"ContainerStarted","Data":"aa085dacb9181d3bef969accfa723226c66de6cb12bed736bab9996b47bc3cc2"}
Oct 30 19:37:27.610342 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:27.610136    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59" event={"ID":"1daee16d-a4f6-4f87-b792-b3866450b2b2","Type":"ContainerStarted","Data":"568c235d3eb781d0bcb5883e2080161b0bd790fe48a24b5f6477cf02dda6a82a"}
Oct 30 19:37:27.610342 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:27.610184    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"
Oct 30 19:37:27.738040 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:27.737965    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59" podStartSLOduration=1.7379468729999998 podStartE2EDuration="1.737946873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:27.736839426 +0000 UTC m=+3020.683007313" watchObservedRunningTime="2026-10-30 19:37:27.737946873 +0000 UTC m=+3020.684114778"
Oct 30 19:37:28.516603 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:28.516571    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:28.738498 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:28.738440    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:28.949604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:28.949527    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:30.125049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:30.124998    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:30.427368 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:30.427259    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:30.618974 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:30.618943    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:33.383881 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:37:33.383844    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:37:33.558160 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:33.558127    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:33.795175 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:33.795142    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:34.010057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:34.009745    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:35.247827 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.247780    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:35.494576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.494538    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:35.532065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.531975    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"]
Oct 30 19:37:35.538325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.538305    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"
Oct 30 19:37:35.647056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.640210    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-85f471-dockercfg-gt9bt\""
Oct 30 19:37:35.647056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.640292    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-85f471\""
Oct 30 19:37:35.668298 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.648766    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c467c6-c1dc-4095-9849-35acde68f3a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62\" (UID: \"f1c467c6-c1dc-4095-9849-35acde68f3a3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"
Oct 30 19:37:35.668298 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.648909    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5m54\" (UniqueName: \"kubernetes.io/projected/f1c467c6-c1dc-4095-9849-35acde68f3a3-kube-api-access-x5m54\") pod \"payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62\" (UID: \"f1c467c6-c1dc-4095-9849-35acde68f3a3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"
Oct 30 19:37:35.668298 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.652761    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"]
Oct 30 19:37:35.730012 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.729973    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"]
Oct 30 19:37:35.735274 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.735005    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"
Oct 30 19:37:35.754435 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.752291    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c467c6-c1dc-4095-9849-35acde68f3a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62\" (UID: \"f1c467c6-c1dc-4095-9849-35acde68f3a3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"
Oct 30 19:37:35.754435 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.752363    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5m54\" (UniqueName: \"kubernetes.io/projected/f1c467c6-c1dc-4095-9849-35acde68f3a3-kube-api-access-x5m54\") pod \"payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62\" (UID: \"f1c467c6-c1dc-4095-9849-35acde68f3a3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"
Oct 30 19:37:35.754435 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.752431    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d434f7-33e5-4136-9114-43d281037737-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2\" (UID: \"20d434f7-33e5-4136-9114-43d281037737\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"
Oct 30 19:37:35.754435 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.752590    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwptf\" (UniqueName: \"kubernetes.io/projected/20d434f7-33e5-4136-9114-43d281037737-kube-api-access-rwptf\") pod \"payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2\" (UID: \"20d434f7-33e5-4136-9114-43d281037737\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"
Oct 30 19:37:35.754435 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.753540    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c467c6-c1dc-4095-9849-35acde68f3a3-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62\" (UID: \"f1c467c6-c1dc-4095-9849-35acde68f3a3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"
Oct 30 19:37:35.811240 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.811184    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5m54\" (UniqueName: \"kubernetes.io/projected/f1c467c6-c1dc-4095-9849-35acde68f3a3-kube-api-access-x5m54\") pod \"payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62\" (UID: \"f1c467c6-c1dc-4095-9849-35acde68f3a3\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"
Oct 30 19:37:35.815783 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.815757    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"]
Oct 30 19:37:35.853748 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.853715    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d434f7-33e5-4136-9114-43d281037737-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2\" (UID: \"20d434f7-33e5-4136-9114-43d281037737\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"
Oct 30 19:37:35.853939 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.853848    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwptf\" (UniqueName: \"kubernetes.io/projected/20d434f7-33e5-4136-9114-43d281037737-kube-api-access-rwptf\") pod \"payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2\" (UID: \"20d434f7-33e5-4136-9114-43d281037737\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"
Oct 30 19:37:35.854085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.854066    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"
Oct 30 19:37:35.854520 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.854496    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d434f7-33e5-4136-9114-43d281037737-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2\" (UID: \"20d434f7-33e5-4136-9114-43d281037737\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"
Oct 30 19:37:35.914024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:35.913469    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwptf\" (UniqueName: \"kubernetes.io/projected/20d434f7-33e5-4136-9114-43d281037737-kube-api-access-rwptf\") pod \"payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2\" (UID: \"20d434f7-33e5-4136-9114-43d281037737\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"
Oct 30 19:37:36.064521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:36.064446    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"
Oct 30 19:37:36.232071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:36.232045    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"]
Oct 30 19:37:36.234529 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:37:36.234501    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1c467c6_c1dc_4095_9849_35acde68f3a3.slice/crio-5c96b6955acb9c83606f1e284f71c6a58fb86c81c7c73fbbfe76e89924ddd4bb WatchSource:0}: Error finding container 5c96b6955acb9c83606f1e284f71c6a58fb86c81c7c73fbbfe76e89924ddd4bb: Status 404 returned error can't find the container with id 5c96b6955acb9c83606f1e284f71c6a58fb86c81c7c73fbbfe76e89924ddd4bb
Oct 30 19:37:36.434600 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:36.434573    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"]
Oct 30 19:37:36.436960 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:37:36.436933    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20d434f7_33e5_4136_9114_43d281037737.slice/crio-cd7e4c63af3b39a068ac79f64943f9c6a9480ad3aac63029891ba707e4f8fd36 WatchSource:0}: Error finding container cd7e4c63af3b39a068ac79f64943f9c6a9480ad3aac63029891ba707e4f8fd36: Status 404 returned error can't find the container with id cd7e4c63af3b39a068ac79f64943f9c6a9480ad3aac63029891ba707e4f8fd36
Oct 30 19:37:36.739004 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:36.738913    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62" event={"ID":"f1c467c6-c1dc-4095-9849-35acde68f3a3","Type":"ContainerStarted","Data":"9d88f0bd77097c3a6120097a075cd90341af362201886527149c18ad8637910d"}
Oct 30 19:37:36.739004 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:36.738961    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62" event={"ID":"f1c467c6-c1dc-4095-9849-35acde68f3a3","Type":"ContainerStarted","Data":"5c96b6955acb9c83606f1e284f71c6a58fb86c81c7c73fbbfe76e89924ddd4bb"}
Oct 30 19:37:36.739263 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:36.739137    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"
Oct 30 19:37:36.740647 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:36.740618    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2" event={"ID":"20d434f7-33e5-4136-9114-43d281037737","Type":"ContainerStarted","Data":"a21ab7d115bde8c9803c85a09ccc758064afceeff6b7caf01d2551950e1c9b8f"}
Oct 30 19:37:36.740781 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:36.740655    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2" event={"ID":"20d434f7-33e5-4136-9114-43d281037737","Type":"ContainerStarted","Data":"cd7e4c63af3b39a068ac79f64943f9c6a9480ad3aac63029891ba707e4f8fd36"}
Oct 30 19:37:36.740781 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:36.740758    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"
Oct 30 19:37:36.853625 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:36.853579    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62" podStartSLOduration=1.853565511 podStartE2EDuration="1.853565511s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:36.851318549 +0000 UTC m=+3029.797486453" watchObservedRunningTime="2026-10-30 19:37:36.853565511 +0000 UTC m=+3029.799733461"
Oct 30 19:37:38.619775 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:38.619749    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"
Oct 30 19:37:38.735706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:38.735652    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2" podStartSLOduration=3.735634452 podStartE2EDuration="3.735634452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:37.009487779 +0000 UTC m=+3029.955656011" watchObservedRunningTime="2026-10-30 19:37:38.735634452 +0000 UTC m=+3031.681802356"
Oct 30 19:37:39.466783 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:39.466735    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:39.700366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:39.700328    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:39.887400 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:39.887361    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:40.225923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:40.224714    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:41.380883 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:41.380853    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:41.566418 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:41.566371    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:44.199127 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.199093    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:44.371225 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.371191    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:44.576263 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.576237    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"]
Oct 30 19:37:44.580469 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.580449    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"
Oct 30 19:37:44.684369 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.684328    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f67a70de-5e78-4a06-a16b-cd846189120d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt\" (UID: \"f67a70de-5e78-4a06-a16b-cd846189120d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"
Oct 30 19:37:44.701408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.684675    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx9qv\" (UniqueName: \"kubernetes.io/projected/f67a70de-5e78-4a06-a16b-cd846189120d-kube-api-access-nx9qv\") pod \"payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt\" (UID: \"f67a70de-5e78-4a06-a16b-cd846189120d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"
Oct 30 19:37:44.726145 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.709326    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-fe7634\""
Oct 30 19:37:44.726145 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.709407    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-fe7634-dockercfg-7q9tq\""
Oct 30 19:37:44.766086 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.763124    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"]
Oct 30 19:37:44.771339 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.771308    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"]
Oct 30 19:37:44.782863 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.782833    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"
Oct 30 19:37:44.802948 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.802919    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f67a70de-5e78-4a06-a16b-cd846189120d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt\" (UID: \"f67a70de-5e78-4a06-a16b-cd846189120d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"
Oct 30 19:37:44.803101 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.803024    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx9qv\" (UniqueName: \"kubernetes.io/projected/f67a70de-5e78-4a06-a16b-cd846189120d-kube-api-access-nx9qv\") pod \"payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt\" (UID: \"f67a70de-5e78-4a06-a16b-cd846189120d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"
Oct 30 19:37:44.803580 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.803558    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f67a70de-5e78-4a06-a16b-cd846189120d-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt\" (UID: \"f67a70de-5e78-4a06-a16b-cd846189120d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"
Oct 30 19:37:44.830245 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.830187    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"]
Oct 30 19:37:44.861367 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.861341    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx9qv\" (UniqueName: \"kubernetes.io/projected/f67a70de-5e78-4a06-a16b-cd846189120d-kube-api-access-nx9qv\") pod \"payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt\" (UID: \"f67a70de-5e78-4a06-a16b-cd846189120d\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"
Oct 30 19:37:44.893716 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.893686    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"
Oct 30 19:37:44.904906 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.904665    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgxxw\" (UniqueName: \"kubernetes.io/projected/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-kube-api-access-mgxxw\") pod \"payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb\" (UID: \"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"
Oct 30 19:37:44.904906 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:44.904798    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb\" (UID: \"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"
Oct 30 19:37:45.006088 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.006027    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgxxw\" (UniqueName: \"kubernetes.io/projected/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-kube-api-access-mgxxw\") pod \"payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb\" (UID: \"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"
Oct 30 19:37:45.006249 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.006202    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb\" (UID: \"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"
Oct 30 19:37:45.006883 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.006864    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb\" (UID: \"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"
Oct 30 19:37:45.064275 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.064252    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgxxw\" (UniqueName: \"kubernetes.io/projected/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-kube-api-access-mgxxw\") pod \"payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb\" (UID: \"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"
Oct 30 19:37:45.086214 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.086021    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:45.097814 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.097779    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"
Oct 30 19:37:45.271726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.271694    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"]
Oct 30 19:37:45.272913 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:37:45.272886    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf67a70de_5e78_4a06_a16b_cd846189120d.slice/crio-82acc4a31de11a39d741f5ee4869e8a13b2ea42f514d11166117b49282d60538 WatchSource:0}: Error finding container 82acc4a31de11a39d741f5ee4869e8a13b2ea42f514d11166117b49282d60538: Status 404 returned error can't find the container with id 82acc4a31de11a39d741f5ee4869e8a13b2ea42f514d11166117b49282d60538
Oct 30 19:37:45.519667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.519640    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"]
Oct 30 19:37:45.521671 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:37:45.521643    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e21e91f_d5eb_4e2b_8ae8_5b47a22732f9.slice/crio-aa31811aa3a11c1105101164721398d17ee61c5c790fb8472ca5f1a220b372d3 WatchSource:0}: Error finding container aa31811aa3a11c1105101164721398d17ee61c5c790fb8472ca5f1a220b372d3: Status 404 returned error can't find the container with id aa31811aa3a11c1105101164721398d17ee61c5c790fb8472ca5f1a220b372d3
Oct 30 19:37:45.962163 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.962124    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb" event={"ID":"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9","Type":"ContainerStarted","Data":"dd9d83fa494b7592f32753c26477bc8fb217ffc4f5f228067b0370e4ddf6020a"}
Oct 30 19:37:45.962375 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.962190    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb" event={"ID":"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9","Type":"ContainerStarted","Data":"aa31811aa3a11c1105101164721398d17ee61c5c790fb8472ca5f1a220b372d3"}
Oct 30 19:37:45.962375 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.962335    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"
Oct 30 19:37:45.963726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.963703    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt" event={"ID":"f67a70de-5e78-4a06-a16b-cd846189120d","Type":"ContainerStarted","Data":"fcb7f418d2c241253a44cf429ac8c39dc0100eae345be6128ab703904f807a9a"}
Oct 30 19:37:45.963866 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.963738    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt" event={"ID":"f67a70de-5e78-4a06-a16b-cd846189120d","Type":"ContainerStarted","Data":"82acc4a31de11a39d741f5ee4869e8a13b2ea42f514d11166117b49282d60538"}
Oct 30 19:37:45.963866 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:45.963833    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"
Oct 30 19:37:46.076690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:46.076645    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb" podStartSLOduration=2.076632828 podStartE2EDuration="2.076632828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:46.07446449 +0000 UTC m=+3039.020632392" watchObservedRunningTime="2026-10-30 19:37:46.076632828 +0000 UTC m=+3039.022800731"
Oct 30 19:37:46.185324 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:46.185281    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt" podStartSLOduration=2.185268301 podStartE2EDuration="2.185268301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:37:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:37:46.184117164 +0000 UTC m=+3039.130285067" watchObservedRunningTime="2026-10-30 19:37:46.185268301 +0000 UTC m=+3039.131436205"
Oct 30 19:37:47.750294 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:47.750265    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"
Oct 30 19:37:47.750721 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:47.750407    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"
Oct 30 19:37:49.678878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:49.678845    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:52.141463 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:52.141432    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:37:54.385526 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:54.385491    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:55.593126 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:55.593091    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:56.365509 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:56.361935    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:56.615176 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:56.615141    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:56.972695 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:56.972665    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"
Oct 30 19:37:56.972853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:56.972747    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"
Oct 30 19:37:59.028333 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:59.028301    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:37:59.230312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:37:59.230273    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:00.664092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:00.664055    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:00.837824 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:00.837791    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:03.383274 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:38:03.383234    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:38:04.182864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:04.182830    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:04.400675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:04.400636    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:04.751974 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:04.751932    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"]
Oct 30 19:38:04.760528 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:04.760496    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"
Oct 30 19:38:04.814221 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:04.814192    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-praxis-a-af72cf-dockercfg-kf8n7\""
Oct 30 19:38:04.843187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:04.843090    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-praxis-a-af72cf\""
Oct 30 19:38:04.845853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:04.845826    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"]
Oct 30 19:38:04.954496 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:04.954464    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqzrl\" (UniqueName: \"kubernetes.io/projected/4ea5bfd4-5c12-462d-beb8-2944091571f5-kube-api-access-lqzrl\") pod \"payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp\" (UID: \"4ea5bfd4-5c12-462d-beb8-2944091571f5\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"
Oct 30 19:38:04.954643 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:04.954528    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ea5bfd4-5c12-462d-beb8-2944091571f5-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp\" (UID: \"4ea5bfd4-5c12-462d-beb8-2944091571f5\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"
Oct 30 19:38:05.055397 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:05.055325    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqzrl\" (UniqueName: \"kubernetes.io/projected/4ea5bfd4-5c12-462d-beb8-2944091571f5-kube-api-access-lqzrl\") pod \"payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp\" (UID: \"4ea5bfd4-5c12-462d-beb8-2944091571f5\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"
Oct 30 19:38:05.055397 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:05.055387    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ea5bfd4-5c12-462d-beb8-2944091571f5-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp\" (UID: \"4ea5bfd4-5c12-462d-beb8-2944091571f5\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"
Oct 30 19:38:05.056062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:05.056020    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ea5bfd4-5c12-462d-beb8-2944091571f5-plugins-config-volume\") pod \"payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp\" (UID: \"4ea5bfd4-5c12-462d-beb8-2944091571f5\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"
Oct 30 19:38:05.113374 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:05.113349    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqzrl\" (UniqueName: \"kubernetes.io/projected/4ea5bfd4-5c12-462d-beb8-2944091571f5-kube-api-access-lqzrl\") pod \"payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp\" (UID: \"4ea5bfd4-5c12-462d-beb8-2944091571f5\") " pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"
Oct 30 19:38:05.116245 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:05.116220    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"
Oct 30 19:38:05.485065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:05.485005    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"]
Oct 30 19:38:05.487590 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:38:05.487554    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea5bfd4_5c12_462d_beb8_2944091571f5.slice/crio-f4f28f8fa03b41afb5aec0d8bcc7353ff3f35eaf08f55238f1c1089e32d9b697 WatchSource:0}: Error finding container f4f28f8fa03b41afb5aec0d8bcc7353ff3f35eaf08f55238f1c1089e32d9b697: Status 404 returned error can't find the container with id f4f28f8fa03b41afb5aec0d8bcc7353ff3f35eaf08f55238f1c1089e32d9b697
Oct 30 19:38:06.111811 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:06.111776    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp" event={"ID":"4ea5bfd4-5c12-462d-beb8-2944091571f5","Type":"ContainerStarted","Data":"a481592aba4ca7c223d9bfd09a9775dba48225fda0d098536c9a49c8fbdb8827"}
Oct 30 19:38:06.112024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:06.111832    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp" event={"ID":"4ea5bfd4-5c12-462d-beb8-2944091571f5","Type":"ContainerStarted","Data":"f4f28f8fa03b41afb5aec0d8bcc7353ff3f35eaf08f55238f1c1089e32d9b697"}
Oct 30 19:38:06.112024 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:06.111867    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"
Oct 30 19:38:06.223016 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:06.222947    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp" podStartSLOduration=2.22293422 podStartE2EDuration="2.22293422s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:06.222062112 +0000 UTC m=+3059.168230012" watchObservedRunningTime="2026-10-30 19:38:06.22293422 +0000 UTC m=+3059.169102124"
Oct 30 19:38:07.009918 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:07.009883    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:09.197102 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:09.197064    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:09.398471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:09.398436    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:09.625438 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:09.625393    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:11.520331 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:11.520288    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:14.021484 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:14.007124    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:17.120264 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:17.120233    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"
Oct 30 19:38:18.686944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:18.686868    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:23.044830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.044794    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:38:23.449819 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.449735    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"]
Oct 30 19:38:23.455707 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.455687    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:23.504711 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.504689    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-fe7634-serving-cert\""
Oct 30 19:38:23.512156 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.512131    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"]
Oct 30 19:38:23.518593 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.518570    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psbr4\" (UniqueName: \"kubernetes.io/projected/2f63052b-ba80-4c6e-9a7c-4413eebb990b-kube-api-access-psbr4\") pod \"maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:23.518720 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.518704    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:23.518785 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.518769    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f63052b-ba80-4c6e-9a7c-4413eebb990b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:23.518839 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.518817    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:23.644675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.644643    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f63052b-ba80-4c6e-9a7c-4413eebb990b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:23.644906 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.644700    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:23.644906 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.644776    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psbr4\" (UniqueName: \"kubernetes.io/projected/2f63052b-ba80-4c6e-9a7c-4413eebb990b-kube-api-access-psbr4\") pod \"maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:23.644906 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:38:23.644857    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-serving-cert: secret "maas-api-e2e-shared-a-w1-fe7634-serving-cert" not found
Oct 30 19:38:23.645162 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.644916    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:23.645162 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:38:23.644958    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-maas-api-tls podName:2f63052b-ba80-4c6e-9a7c-4413eebb990b nodeName:}" failed. No retries permitted until 2026-10-30 19:38:24.14493376 +0000 UTC m=+3077.091101643 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-maas-api-tls") pod "maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6" (UID: "2f63052b-ba80-4c6e-9a7c-4413eebb990b") : secret "maas-api-e2e-shared-a-w1-fe7634-serving-cert" not found
Oct 30 19:38:23.645463 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.645441    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f63052b-ba80-4c6e-9a7c-4413eebb990b-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:23.647505 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.647482    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:23.674358 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.674330    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:23.772053 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.758759    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-psbr4\" (UniqueName: \"kubernetes.io/projected/2f63052b-ba80-4c6e-9a7c-4413eebb990b-kube-api-access-psbr4\") pod \"maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:23.918811 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:23.918769    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:24.093392 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:24.093297    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:24.172196 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:24.172167    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:24.174506 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:24.174483    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:24.370533 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:24.370507    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:24.754097 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:24.754070    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"]
Oct 30 19:38:24.758024 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:38:24.757985    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f63052b_ba80_4c6e_9a7c_4413eebb990b.slice/crio-07ab22bc8419c017c65d62a4dc914f060ad275734aa874b516405921da11474a WatchSource:0}: Error finding container 07ab22bc8419c017c65d62a4dc914f060ad275734aa874b516405921da11474a: Status 404 returned error can't find the container with id 07ab22bc8419c017c65d62a4dc914f060ad275734aa874b516405921da11474a
Oct 30 19:38:24.759829 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:24.759810    5704 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:38:24.878712 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:24.878676    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:25.253066 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:25.253001    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6" event={"ID":"2f63052b-ba80-4c6e-9a7c-4413eebb990b","Type":"ContainerStarted","Data":"07ab22bc8419c017c65d62a4dc914f060ad275734aa874b516405921da11474a"}
Oct 30 19:38:25.459144 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:25.459099    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:26.052677 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:26.052641    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:26.280147 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:26.280107    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6" event={"ID":"2f63052b-ba80-4c6e-9a7c-4413eebb990b","Type":"ContainerStarted","Data":"ca77e355714ec0cd5a4744f9b09c72e5a72cb9dab6a5c3ebd972c437cc594ff8"}
Oct 30 19:38:26.289068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:26.281208    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:26.379727 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:26.379642    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:26.397366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:26.397303    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6" podStartSLOduration=2.555010954 podStartE2EDuration="3.39728388s" totalImagesPullingTime="842.272926ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:26.393923181 +0000 UTC m=+3079.340091124" watchObservedRunningTime="2026-10-30 19:38:26.39728388 +0000 UTC m=+3079.343451784"
Oct 30 19:38:27.395196 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:27.395148    5704 patch_prober.go:29] interesting pod/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:38:27.395636 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:27.395223    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:38:29.029062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:29.025568    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:29.345716 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:29.345632    5704 generic.go:356] "Generic (PLEG): container finished" podID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerID="62a43352f0135bbcbc52dbabfea0482e953e6ae252895e875f2c94edeed4b790" exitCode=0
Oct 30 19:38:29.345716 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:29.345686    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" event={"ID":"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce","Type":"ContainerDied","Data":"62a43352f0135bbcbc52dbabfea0482e953e6ae252895e875f2c94edeed4b790"}
Oct 30 19:38:29.346188 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:29.346171    5704 scope.go:122] "RemoveContainer" containerID="62a43352f0135bbcbc52dbabfea0482e953e6ae252895e875f2c94edeed4b790"
Oct 30 19:38:29.347835 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:29.347803    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:38:29.347925 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:29.347884    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:38:29.347925 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:29.347916    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:38:30.355456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:30.355422    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" event={"ID":"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce","Type":"ContainerStarted","Data":"2fe03da0147966fb1e6b653c0505b5ffa0d0d0945e691ea376ba312f0aaf31a5"}
Oct 30 19:38:31.287218 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:31.287185    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:38:31.287397 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:31.287349    5704 patch_prober.go:29] interesting pod/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:38:31.287448 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:31.287394    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 19:38:32.286878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:32.286841    5704 patch_prober.go:29] interesting pod/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:38:32.287303 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:32.286907    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 19:38:32.545318 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:32.545217    5704 patch_prober.go:29] interesting pod/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Oct 30 19:38:32.545318 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:32.545285    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:38:33.286550 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:33.286516    5704 patch_prober.go:29] interesting pod/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:38:33.286733 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:33.286580    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 19:38:33.322509 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:33.321464    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:38:33.384304 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:38:33.384254    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:38:33.671256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:33.671158    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:34.287326 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:34.287293    5704 patch_prober.go:29] interesting pod/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:38:34.287596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:34.287352    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 19:38:35.286446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:35.286418    5704 patch_prober.go:29] interesting pod/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused" start-of-body=
Oct 30 19:38:35.286895 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:35.286467    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.142:15021/healthz/ready\": dial tcp 10.134.0.142:15021: connect: connection refused"
Oct 30 19:38:35.391779 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:35.391747    5704 generic.go:356] "Generic (PLEG): container finished" podID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerID="47071d4e5aea22e8e246f9949e8093c5068faf1195413c78c5d5cd85fa934b5e" exitCode=0
Oct 30 19:38:35.391949 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:35.391818    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" event={"ID":"139cc6a4-643f-41ee-a8c2-eaf2177f4052","Type":"ContainerDied","Data":"47071d4e5aea22e8e246f9949e8093c5068faf1195413c78c5d5cd85fa934b5e"}
Oct 30 19:38:35.392206 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:35.392191    5704 scope.go:122] "RemoveContainer" containerID="47071d4e5aea22e8e246f9949e8093c5068faf1195413c78c5d5cd85fa934b5e"
Oct 30 19:38:35.393567 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:35.393540    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:38:35.393629 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:35.393608    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:38:35.393667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:35.393640    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:38:36.290468 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:36.290435    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:38:36.290849 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:36.290798    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:38:36.291616 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:36.291597    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:38:36.400529 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:36.400498    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" event={"ID":"139cc6a4-643f-41ee-a8c2-eaf2177f4052","Type":"ContainerStarted","Data":"aced46024a8e2981a595d19d08043040ac3b9eab0611779ca4dc98a53536bdb2"}
Oct 30 19:38:37.339138 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:37.339096    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:38:37.339756 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:37.339194    5704 patch_prober.go:29] interesting pod/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:38:37.339955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:37.339923    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:38:38.360100 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:38.353022    5704 patch_prober.go:29] interesting pod/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:38:38.360100 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:38.353099    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:38:38.370057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:38.366801    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:39.331391 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:39.331356    5704 patch_prober.go:29] interesting pod/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:38:39.331675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:39.331646    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:38:40.047803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:40.047770    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:40.240057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:40.239565    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:40.331116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:40.325405    5704 patch_prober.go:29] interesting pod/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:38:40.331116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:40.325467    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:38:41.340667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:41.337088    5704 patch_prober.go:29] interesting pod/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused" start-of-body=
Oct 30 19:38:41.340667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:41.337147    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.145:15021/healthz/ready\": dial tcp 10.134.0.145:15021: connect: connection refused"
Oct 30 19:38:42.325497 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:42.325462    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:38:42.325842 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:42.325824    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:38:42.326596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:42.326574    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:38:43.134627 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.134590    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:43.320382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.320342    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:43.504756 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.504727    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:43.705056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.700599    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"]
Oct 30 19:38:43.705411 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.705332    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"
Oct 30 19:38:43.759107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.736475    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"]
Oct 30 19:38:43.797811 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.797786    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w3-b2604c-dockercfg-97tj7\""
Oct 30 19:38:43.799533 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.799515    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w3-b2604c\""
Oct 30 19:38:43.839404 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.839377    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f628790-151a-4760-9dfd-62f0ca9c87e4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64\" (UID: \"0f628790-151a-4760-9dfd-62f0ca9c87e4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"
Oct 30 19:38:43.839557 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.839426    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zcmk\" (UniqueName: \"kubernetes.io/projected/0f628790-151a-4760-9dfd-62f0ca9c87e4-kube-api-access-5zcmk\") pod \"payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64\" (UID: \"0f628790-151a-4760-9dfd-62f0ca9c87e4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"
Oct 30 19:38:43.940634 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.940591    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f628790-151a-4760-9dfd-62f0ca9c87e4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64\" (UID: \"0f628790-151a-4760-9dfd-62f0ca9c87e4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"
Oct 30 19:38:43.940634 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.940639    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zcmk\" (UniqueName: \"kubernetes.io/projected/0f628790-151a-4760-9dfd-62f0ca9c87e4-kube-api-access-5zcmk\") pod \"payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64\" (UID: \"0f628790-151a-4760-9dfd-62f0ca9c87e4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"
Oct 30 19:38:43.941456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.941429    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f628790-151a-4760-9dfd-62f0ca9c87e4-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64\" (UID: \"0f628790-151a-4760-9dfd-62f0ca9c87e4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"
Oct 30 19:38:43.998810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:43.998783    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zcmk\" (UniqueName: \"kubernetes.io/projected/0f628790-151a-4760-9dfd-62f0ca9c87e4-kube-api-access-5zcmk\") pod \"payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64\" (UID: \"0f628790-151a-4760-9dfd-62f0ca9c87e4\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"
Oct 30 19:38:44.105975 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:44.105912    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"
Oct 30 19:38:44.474881 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:44.474779    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"]
Oct 30 19:38:44.477904 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:38:44.477877    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f628790_151a_4760_9dfd_62f0ca9c87e4.slice/crio-962f7954f9a94f1ed1ec3f79c8ecdc84458d97da2e7325cbf52176cee9bf2f08 WatchSource:0}: Error finding container 962f7954f9a94f1ed1ec3f79c8ecdc84458d97da2e7325cbf52176cee9bf2f08: Status 404 returned error can't find the container with id 962f7954f9a94f1ed1ec3f79c8ecdc84458d97da2e7325cbf52176cee9bf2f08
Oct 30 19:38:44.490353 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:44.490323    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64" event={"ID":"0f628790-151a-4760-9dfd-62f0ca9c87e4","Type":"ContainerStarted","Data":"962f7954f9a94f1ed1ec3f79c8ecdc84458d97da2e7325cbf52176cee9bf2f08"}
Oct 30 19:38:45.498812 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:45.498775    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64" event={"ID":"0f628790-151a-4760-9dfd-62f0ca9c87e4","Type":"ContainerStarted","Data":"2c5e10ea1c27d6acbadb8fb6da10aa670bdaf415e9e6b6ed34972fa193c067f3"}
Oct 30 19:38:45.499240 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:45.498981    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"
Oct 30 19:38:45.611261 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:45.611204    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64" podStartSLOduration=2.611190598 podStartE2EDuration="2.611190598s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:45.609554463 +0000 UTC m=+3098.555722368" watchObservedRunningTime="2026-10-30 19:38:45.611190598 +0000 UTC m=+3098.557358500"
Oct 30 19:38:47.228754 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:47.228720    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:38:48.182063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:48.173864    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:48.363105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:48.363067    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:48.610971 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:48.610940    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:52.925273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:52.921551    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:56.508346 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:56.508315    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"
Oct 30 19:38:57.635904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:57.635859    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:38:59.679298 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:38:59.679260    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:00.227126 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.227090    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:00.689735 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.689704    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"]
Oct 30 19:39:00.693845 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.693821    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.743180 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.743103    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w0-d6f58f-openshift-default-dockercfg-wqt9q\""
Oct 30 19:39:00.798868 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.798838    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"]
Oct 30 19:39:00.804306 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.804275    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-istio-token\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.804460 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.804329    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcgtm\" (UniqueName: \"kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-kube-api-access-zcgtm\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.804505 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.804476    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-credential-socket\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.804542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.804503    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/902a14d3-7e9d-448c-a951-465010109b2c-istiod-ca-cert\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.804542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.804533    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-socket\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.804610 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.804563    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-data\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.804647 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.804615    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-envoy\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.804681 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.804659    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/902a14d3-7e9d-448c-a951-465010109b2c-istio-podinfo\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.804716 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.804682    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-certs\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.813707 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.813656    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:00.912057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.908374    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-istio-token\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.912057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.908560    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcgtm\" (UniqueName: \"kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-kube-api-access-zcgtm\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.912057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.908700    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-credential-socket\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.912057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.908736    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/902a14d3-7e9d-448c-a951-465010109b2c-istiod-ca-cert\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.912057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.908801    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-socket\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.912057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.908873    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-data\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.912057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.908952    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-envoy\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.912057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.909022    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/902a14d3-7e9d-448c-a951-465010109b2c-istio-podinfo\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.912057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.909091    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-certs\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.912057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.909494    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-certs\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.937065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.935857    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-data\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.937263 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.937245    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/902a14d3-7e9d-448c-a951-465010109b2c-istiod-ca-cert\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.942558 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.937587    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-socket\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.964388 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.964303    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-envoy\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.971919 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.971877    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/902a14d3-7e9d-448c-a951-465010109b2c-istio-podinfo\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.972241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.972216    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-credential-socket\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.973223 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.973168    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-istio-token\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:00.975107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:00.975064    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcgtm\" (UniqueName: \"kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-kube-api-access-zcgtm\") pod \"e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") " pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:01.006844 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:01.006818    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:01.365889 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:01.365830    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:01.442111 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:01.442073    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"]
Oct 30 19:39:01.444428 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:39:01.444397    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod902a14d3_7e9d_448c_a951_465010109b2c.slice/crio-9a78a34269a52a4fa652ad1b6641e462205bb1b52699cab16ce40fb491e2dace WatchSource:0}: Error finding container 9a78a34269a52a4fa652ad1b6641e462205bb1b52699cab16ce40fb491e2dace: Status 404 returned error can't find the container with id 9a78a34269a52a4fa652ad1b6641e462205bb1b52699cab16ce40fb491e2dace
Oct 30 19:39:01.447014 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:01.446977    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:01.447124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:01.447056    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:01.447124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:01.447090    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:39:01.598649 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:01.598613    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp" event={"ID":"902a14d3-7e9d-448c-a951-465010109b2c","Type":"ContainerStarted","Data":"12164d02a5113750f6074c2c1120775156b0f0514e8ddd80b85757c005dfe0eb"}
Oct 30 19:39:01.598820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:01.598674    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp" event={"ID":"902a14d3-7e9d-448c-a951-465010109b2c","Type":"ContainerStarted","Data":"9a78a34269a52a4fa652ad1b6641e462205bb1b52699cab16ce40fb491e2dace"}
Oct 30 19:39:01.712925 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:01.712811    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp" podStartSLOduration=1.712797593 podStartE2EDuration="1.712797593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:01.7125999 +0000 UTC m=+3114.658767791" watchObservedRunningTime="2026-10-30 19:39:01.712797593 +0000 UTC m=+3114.658965495"
Oct 30 19:39:02.007320 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:02.007292    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:02.012080 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:02.012059    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:02.564650 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:02.564622    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:02.631124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:02.631082    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:02.632632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:02.632612    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:39:03.383690 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:39:03.383651    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:39:04.647595 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:04.647562    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:04.829048 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:04.829003    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:07.644276 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:07.644238    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:07.894813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:07.894704    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:08.076394 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:08.076355    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:10.769156 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:10.769121    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:11.237356 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:11.237263    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:11.879698 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:11.879667    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:12.057579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:12.057540    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:12.230452 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:12.230375    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:12.761253 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:12.761215    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:17.510663 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:17.505734    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:22.301634 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:22.301535    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:26.864269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:26.864235    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:27.074087 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:27.074022    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:27.361571 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:27.361510    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:30.021915 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:30.021872    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:30.204959 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:30.204878    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:30.253617 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:30.253588    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:39:31.800952 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:31.800921    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"]
Oct 30 19:39:31.805008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:31.804987    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:31.854847 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:31.854825    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-b2604c-serving-cert\""
Oct 30 19:39:31.906098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:31.903410    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:31.906098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:31.903526    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/479084c0-041e-4bf9-b2ce-0721d9fb3f75-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:31.906098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:31.903567    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:31.906098 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:31.903779    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztbrj\" (UniqueName: \"kubernetes.io/projected/479084c0-041e-4bf9-b2ce-0721d9fb3f75-kube-api-access-ztbrj\") pod \"maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:31.910658 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:31.910619    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"]
Oct 30 19:39:32.004984 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:32.004952    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/479084c0-041e-4bf9-b2ce-0721d9fb3f75-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:32.005201 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:32.004994    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:32.005285 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:32.005230    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztbrj\" (UniqueName: \"kubernetes.io/projected/479084c0-041e-4bf9-b2ce-0721d9fb3f75-kube-api-access-ztbrj\") pod \"maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:32.005347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:32.005298    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:32.005440 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:39:32.005415    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-serving-cert: secret "maas-api-e2e-shared-b-w3-b2604c-serving-cert" not found
Oct 30 19:39:32.005530 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:39:32.005518    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-maas-api-tls podName:479084c0-041e-4bf9-b2ce-0721d9fb3f75 nodeName:}" failed. No retries permitted until 2026-10-30 19:39:32.505498708 +0000 UTC m=+3145.451666597 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-maas-api-tls") pod "maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb" (UID: "479084c0-041e-4bf9-b2ce-0721d9fb3f75") : secret "maas-api-e2e-shared-b-w3-b2604c-serving-cert" not found
Oct 30 19:39:32.005725 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:32.005698    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/479084c0-041e-4bf9-b2ce-0721d9fb3f75-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:32.007621 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:32.007604    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:32.023678 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:32.023650    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:32.079014 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:32.074971    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztbrj\" (UniqueName: \"kubernetes.io/projected/479084c0-041e-4bf9-b2ce-0721d9fb3f75-kube-api-access-ztbrj\") pod \"maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:32.299270 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:32.299237    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:32.476968 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:32.476888    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:32.521397 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:32.521363    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:32.523855 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:32.523834    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:32.718195 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:32.718166    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:33.088640 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:33.088617    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"]
Oct 30 19:39:33.091312 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:39:33.091274    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod479084c0_041e_4bf9_b2ce_0721d9fb3f75.slice/crio-0a7d744065bb0bd2ec37e8cc5605b0c2a1d0998229c1daf85cf774fac363bd4c WatchSource:0}: Error finding container 0a7d744065bb0bd2ec37e8cc5605b0c2a1d0998229c1daf85cf774fac363bd4c: Status 404 returned error can't find the container with id 0a7d744065bb0bd2ec37e8cc5605b0c2a1d0998229c1daf85cf774fac363bd4c
Oct 30 19:39:33.383430 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:39:33.383346    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:39:33.934332 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:33.934302    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb" event={"ID":"479084c0-041e-4bf9-b2ce-0721d9fb3f75","Type":"ContainerStarted","Data":"0a7d744065bb0bd2ec37e8cc5605b0c2a1d0998229c1daf85cf774fac363bd4c"}
Oct 30 19:39:34.941973 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:34.941938    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb" event={"ID":"479084c0-041e-4bf9-b2ce-0721d9fb3f75","Type":"ContainerStarted","Data":"5228733e421ea710909e671e8b0a643b01146085a3884967ab164cbc7e56bcba"}
Oct 30 19:39:34.942364 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:34.942166    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:35.055268 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:35.055224    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb" podStartSLOduration=3.244798253 podStartE2EDuration="4.055210473s" totalImagesPullingTime="810.41222ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:39:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:39:35.053445622 +0000 UTC m=+3147.999613527" watchObservedRunningTime="2026-10-30 19:39:35.055210473 +0000 UTC m=+3148.001378375"
Oct 30 19:39:36.334863 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:36.334778    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:36.768728 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:36.768690    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:36.950962 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:36.950932    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:37.474091 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:37.463217    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:37.684278 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:37.684245    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:38.735398 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:38.735362    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:40.955170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:40.955143    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:39:41.561484 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:41.560761    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:41.835706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:41.835626    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:42.015433 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:42.015401    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:42.229319 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:42.229237    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:44.943209 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:44.943174    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:45.159808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:45.159770    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:46.634617 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:46.634578    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:51.410109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:51.408140    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:52.761054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:52.757804    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:39:56.311757 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:39:56.311700    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:01.376835 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:01.376794    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:03.384487 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:40:03.384450    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:40:04.496185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.496151    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"]
Oct 30 19:40:04.499907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.499885    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.549426 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.549390    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-praxis-b-a2f378-openshift-default-dockercfg-tnk4l\""
Oct 30 19:40:04.630054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.604955    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-envoy\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.630315 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.623948    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"]
Oct 30 19:40:04.641169 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.630461    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-credential-socket\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.641457 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.641437    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istiod-ca-cert\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.651290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.651234    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-token\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.651464 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.651345    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-certs\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.651464 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.651453    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-data\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.651593 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.651577    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdzk6\" (UniqueName: \"kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-kube-api-access-kdzk6\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.651653 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.651631    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-podinfo\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.651747 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.651730    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-socket\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.753070 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.752941    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdzk6\" (UniqueName: \"kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-kube-api-access-kdzk6\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.753248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.753081    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-podinfo\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.753248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.753191    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-socket\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.753248 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.753242    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-envoy\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.753390 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.753269    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-credential-socket\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.753390 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.753300    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istiod-ca-cert\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.753390 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.753371    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-token\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.753507 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.753422    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-certs\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.753507 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.753481    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-data\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.754029 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.753980    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-socket\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.754187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.754162    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-credential-socket\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.754321 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.754301    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-certs\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.754567 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.754547    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-data\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.754674 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.754562    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istiod-ca-cert\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.755968 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.755927    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-envoy\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.756197 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.756174    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-podinfo\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.811488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.811460    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-token\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.811618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.811560    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdzk6\" (UniqueName: \"kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-kube-api-access-kdzk6\") pod \"e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") " pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:04.815732 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:04.815713    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:05.145314 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:05.145232    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:05.240699 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:05.240676    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"]
Oct 30 19:40:05.243438 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:40:05.243412    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb4ec43_aa95_4bad_b9fb_71fb87308901.slice/crio-c783a189681f7ec7c2e53d3f326a9450055c74659d23af89ca34ee229b970fc9 WatchSource:0}: Error finding container c783a189681f7ec7c2e53d3f326a9450055c74659d23af89ca34ee229b970fc9: Status 404 returned error can't find the container with id c783a189681f7ec7c2e53d3f326a9450055c74659d23af89ca34ee229b970fc9
Oct 30 19:40:05.245910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:05.245877    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:05.245976 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:05.245942    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:05.245976 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:05.245972    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:05.330116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:05.330085    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:06.036338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:06.036310    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:06.178246 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:06.178218    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6" event={"ID":"bcb4ec43-aa95-4bad-b9fb-71fb87308901","Type":"ContainerStarted","Data":"f19556b3af32002a6f6999037c4200fb21c5d1c629791e2c5cad0605cfedb4a0"}
Oct 30 19:40:06.178436 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:06.178257    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6" event={"ID":"bcb4ec43-aa95-4bad-b9fb-71fb87308901","Type":"ContainerStarted","Data":"c783a189681f7ec7c2e53d3f326a9450055c74659d23af89ca34ee229b970fc9"}
Oct 30 19:40:06.214366 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:06.214334    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:06.352589 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:06.352500    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6" podStartSLOduration=2.352487336 podStartE2EDuration="2.352487336s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:06.35202665 +0000 UTC m=+3179.298194555" watchObservedRunningTime="2026-10-30 19:40:06.352487336 +0000 UTC m=+3179.298655238"
Oct 30 19:40:06.385544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:06.385506    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:06.816828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:06.816800    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:06.822931 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:06.822905    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:07.183682 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:07.183611    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:07.184640 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:07.184620    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:40:09.200619 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:09.200582    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:09.437692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:09.437652    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:11.199610 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:11.199574    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:12.240432 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:12.240400    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:13.961687 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:13.961653    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:14.193494 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:14.193465    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:15.981503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:15.981469    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:19.146142 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:19.146106    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:40:20.604469 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:20.604430    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:25.167193 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:25.167158    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:29.410111 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.410076    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"]
Oct 30 19:40:29.414234 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.414211    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:29.417563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.417540    5704 generic.go:356] "Generic (PLEG): container finished" podID="902a14d3-7e9d-448c-a951-465010109b2c" containerID="12164d02a5113750f6074c2c1120775156b0f0514e8ddd80b85757c005dfe0eb" exitCode=0
Oct 30 19:40:29.417685 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.417624    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp" event={"ID":"902a14d3-7e9d-448c-a951-465010109b2c","Type":"ContainerDied","Data":"12164d02a5113750f6074c2c1120775156b0f0514e8ddd80b85757c005dfe0eb"}
Oct 30 19:40:29.418009 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.417995    5704 scope.go:122] "RemoveContainer" containerID="12164d02a5113750f6074c2c1120775156b0f0514e8ddd80b85757c005dfe0eb"
Oct 30 19:40:29.419572 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.419542    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:29.419656 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.419602    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:29.419656 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.419635    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:40:29.465453 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.465422    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-c01685-serving-cert\""
Oct 30 19:40:29.469195 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.469175    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9af97f5-65b8-415a-847c-ee8fb9e67da5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:29.469313 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.469293    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:29.469385 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.469352    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:29.469537 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.469512    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpg5g\" (UniqueName: \"kubernetes.io/projected/c9af97f5-65b8-415a-847c-ee8fb9e67da5-kube-api-access-wpg5g\") pod \"maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:29.514754 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.514728    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"]
Oct 30 19:40:29.570452 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.570419    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpg5g\" (UniqueName: \"kubernetes.io/projected/c9af97f5-65b8-415a-847c-ee8fb9e67da5-kube-api-access-wpg5g\") pod \"maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:29.570640 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.570595    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9af97f5-65b8-415a-847c-ee8fb9e67da5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:29.570769 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.570667    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:29.570769 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.570728    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:29.570947 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:40:29.570928    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-serving-cert: secret "maas-api-e2e-shared-b-w2-c01685-serving-cert" not found
Oct 30 19:40:29.571018 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:40:29.570994    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-maas-api-tls podName:c9af97f5-65b8-415a-847c-ee8fb9e67da5 nodeName:}" failed. No retries permitted until 2026-10-30 19:40:30.070972939 +0000 UTC m=+3203.017140837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-maas-api-tls") pod "maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl" (UID: "c9af97f5-65b8-415a-847c-ee8fb9e67da5") : secret "maas-api-e2e-shared-b-w2-c01685-serving-cert" not found
Oct 30 19:40:29.571873 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.571818    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9af97f5-65b8-415a-847c-ee8fb9e67da5-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:29.573443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.573422    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:29.627962 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.627934    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpg5g\" (UniqueName: \"kubernetes.io/projected/c9af97f5-65b8-415a-847c-ee8fb9e67da5-kube-api-access-wpg5g\") pod \"maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:29.640067 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.640043    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:29.831059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:29.824730    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:30.018854 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:30.018817    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:30.101748 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:30.101664    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:30.104131 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:30.104103    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:30.336640 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:30.336276    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:30.457447 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:30.457010    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp" event={"ID":"902a14d3-7e9d-448c-a951-465010109b2c","Type":"ContainerStarted","Data":"fb8946f6638ff867099866d0b9b23efaf0535cbfa40c1d261356d606f9e53291"}
Oct 30 19:40:30.712303 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:30.712280    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"]
Oct 30 19:40:30.714722 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:40:30.714692    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9af97f5_65b8_415a_847c_ee8fb9e67da5.slice/crio-1dccc39a46597208e94fcd38d001df42bafdd31a7601a1bca75ce1a292cdf732 WatchSource:0}: Error finding container 1dccc39a46597208e94fcd38d001df42bafdd31a7601a1bca75ce1a292cdf732: Status 404 returned error can't find the container with id 1dccc39a46597208e94fcd38d001df42bafdd31a7601a1bca75ce1a292cdf732
Oct 30 19:40:31.007846 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:31.007807    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:40:31.008706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:31.008684    5704 patch_prober.go:29] interesting pod/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 19:40:31.008834 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:31.008737    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp" podUID="902a14d3-7e9d-448c-a951-465010109b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 19:40:31.506905 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:31.506820    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl" event={"ID":"c9af97f5-65b8-415a-847c-ee8fb9e67da5","Type":"ContainerStarted","Data":"1dccc39a46597208e94fcd38d001df42bafdd31a7601a1bca75ce1a292cdf732"}
Oct 30 19:40:32.007909 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:32.007879    5704 patch_prober.go:29] interesting pod/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 19:40:32.008102 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:32.007932    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp" podUID="902a14d3-7e9d-448c-a951-465010109b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 19:40:32.514310 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:32.514267    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl" event={"ID":"c9af97f5-65b8-415a-847c-ee8fb9e67da5","Type":"ContainerStarted","Data":"36ca8f7d28461dbed5bc293fe8a50a0600feb2818c990e46ed7069245ff7cc8e"}
Oct 30 19:40:32.514708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:32.514336    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:32.641398 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:32.641364    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:40:32.643655 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:32.643594    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl" podStartSLOduration=2.77166367 podStartE2EDuration="3.643575706s" totalImagesPullingTime="871.912036ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:40:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:40:32.641304877 +0000 UTC m=+3205.587472783" watchObservedRunningTime="2026-10-30 19:40:32.643575706 +0000 UTC m=+3205.589743610"
Oct 30 19:40:33.007391 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:33.007365    5704 patch_prober.go:29] interesting pod/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 19:40:33.007533 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:33.007416    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp" podUID="902a14d3-7e9d-448c-a951-465010109b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 19:40:33.384532 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:40:33.384449    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:40:34.007289 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:34.007260    5704 patch_prober.go:29] interesting pod/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 19:40:34.007703 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:34.007316    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp" podUID="902a14d3-7e9d-448c-a951-465010109b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 19:40:34.452542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:34.452454    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:35.008113 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:35.008078    5704 patch_prober.go:29] interesting pod/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused" start-of-body=
Oct 30 19:40:35.008502 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:35.008136    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp" podUID="902a14d3-7e9d-448c-a951-465010109b2c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.154:15021/healthz/ready\": dial tcp 10.134.0.154:15021: connect: connection refused"
Oct 30 19:40:36.011852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:36.011822    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:40:36.012741 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:36.012719    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:40:38.542766 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:38.542735    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:40:39.115057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:39.114731    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:43.732058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:43.724746    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:46.446149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:46.446113    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:40:47.801104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:47.801068    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:48.661285 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:48.661247    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:48.894170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:48.894135    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:49.073765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:49.073735    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:51.441288 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:51.441252    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:54.020513 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:54.020477    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:40:58.725063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:40:58.724590    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:03.384205 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:41:03.384173    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:41:03.410173 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:03.410143    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:08.064382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:08.064349    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:12.556727 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:12.556695    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:13.570767 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:13.570703    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:13.796041 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:13.796005    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:41:17.642554 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:17.642500    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:22.005736 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.005704    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"]
Oct 30 19:41:22.009625 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.009605    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.058316 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.058294    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-praxis-b-a2f378-serving-cert\""
Oct 30 19:41:22.109243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.109213    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"]
Oct 30 19:41:22.111725 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.111698    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-maas-api-tls\") pod \"maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.111878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.111862    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spc2p\" (UniqueName: \"kubernetes.io/projected/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-kube-api-access-spc2p\") pod \"maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.111943 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.111931    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.112104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.112081    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-metrics-certs\") pod \"maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.212968 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.212937    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spc2p\" (UniqueName: \"kubernetes.io/projected/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-kube-api-access-spc2p\") pod \"maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.213145 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.213014    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.213145 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.213108    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-metrics-certs\") pod \"maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.213244 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.213154    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-maas-api-tls\") pod \"maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.213325 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:41:22.213307    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-serving-cert: secret "maas-api-e2e-praxis-b-a2f378-serving-cert" not found
Oct 30 19:41:22.213390 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:41:22.213383    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-maas-api-tls podName:0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b nodeName:}" failed. No retries permitted until 2026-10-30 19:41:22.713361213 +0000 UTC m=+3255.659529096 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-maas-api-tls") pod "maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl" (UID: "0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b") : secret "maas-api-e2e-praxis-b-a2f378-serving-cert" not found
Oct 30 19:41:22.213763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.213743    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-openshift-service-ca\") pod \"maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.215657 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.215637    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-metrics-certs\") pod \"maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.267777 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.267666    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:22.279319 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.279226    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spc2p\" (UniqueName: \"kubernetes.io/projected/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-kube-api-access-spc2p\") pod \"maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.463054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.460150    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:22.640309 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.640212    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:22.720118 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.720085    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-maas-api-tls\") pod \"maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.722641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.722610    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-maas-api-tls\") pod \"maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") " pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.922944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.922861    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:22.987280 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:22.986523    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:23.330521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:23.330499    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"]
Oct 30 19:41:23.332422 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:41:23.332398    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0af5e3bf_9e24_4a4a_a591_f2dbda8ccd5b.slice/crio-4b471e964d5ef7320aaa6b4e74ffb1488f5640fe9a837de992497bed795a87b6 WatchSource:0}: Error finding container 4b471e964d5ef7320aaa6b4e74ffb1488f5640fe9a837de992497bed795a87b6: Status 404 returned error can't find the container with id 4b471e964d5ef7320aaa6b4e74ffb1488f5640fe9a837de992497bed795a87b6
Oct 30 19:41:23.908245 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:23.908209    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl" event={"ID":"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b","Type":"ContainerStarted","Data":"4b471e964d5ef7320aaa6b4e74ffb1488f5640fe9a837de992497bed795a87b6"}
Oct 30 19:41:24.917452 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:24.917415    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl" event={"ID":"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b","Type":"ContainerStarted","Data":"d50836f917af1fb4b98c18f3ec55318939ed33e58fb72f6b04ed9a43d9532be1"}
Oct 30 19:41:24.917923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:24.917476    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:25.054085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:25.053491    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl" podStartSLOduration=3.265286982 podStartE2EDuration="4.053470518s" totalImagesPullingTime="788.183536ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:41:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:25.034961053 +0000 UTC m=+3257.981128957" watchObservedRunningTime="2026-10-30 19:41:25.053470518 +0000 UTC m=+3257.999638422"
Oct 30 19:41:27.341319 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:27.341282    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:30.929308 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:30.929281    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:41:32.014857 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:32.014816    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:32.278788 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:32.278698    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:33.383790 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:41:33.383756    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:41:37.008062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:36.998233    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:41.269593 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:41.269539    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:41.610219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:41.610107    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:46.226942 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:46.226904    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:49.285790 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:49.285755    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:41:50.879695 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:50.879641    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:53.145607 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:53.145563    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:54.878304 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:54.878275    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:55.743342 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:55.743056    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:41:59.274384 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:59.274349    5704 generic.go:356] "Generic (PLEG): container finished" podID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerID="f19556b3af32002a6f6999037c4200fb21c5d1c629791e2c5cad0605cfedb4a0" exitCode=0
Oct 30 19:41:59.274816 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:59.274421    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6" event={"ID":"bcb4ec43-aa95-4bad-b9fb-71fb87308901","Type":"ContainerDied","Data":"f19556b3af32002a6f6999037c4200fb21c5d1c629791e2c5cad0605cfedb4a0"}
Oct 30 19:41:59.274887 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:59.274873    5704 scope.go:122] "RemoveContainer" containerID="f19556b3af32002a6f6999037c4200fb21c5d1c629791e2c5cad0605cfedb4a0"
Oct 30 19:41:59.276569 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:59.276540    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:41:59.276635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:59.276610    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:41:59.276671 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:41:59.276639    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:42:00.282871 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:00.282830    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6" event={"ID":"bcb4ec43-aa95-4bad-b9fb-71fb87308901","Type":"ContainerStarted","Data":"bd1b56426fbe99b3d7ecf399ea845cfb10dacd1c1e6a633982fe1126efbc89a1"}
Oct 30 19:42:00.347393 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:00.345591    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:00.816451 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:00.816416    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:42:00.816637 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:00.816557    5704 patch_prober.go:29] interesting pod/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:42:00.816637 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:00.816618    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6" podUID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Oct 30 19:42:01.816627 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:01.816592    5704 patch_prober.go:29] interesting pod/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:42:01.817056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:01.816655    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6" podUID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Oct 30 19:42:02.816712 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:02.816674    5704 patch_prober.go:29] interesting pod/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:42:02.817194 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:02.816738    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6" podUID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Oct 30 19:42:03.384142 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:42:03.384104    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:42:03.816708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:03.816679    5704 patch_prober.go:29] interesting pod/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:42:03.816891 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:03.816730    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6" podUID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Oct 30 19:42:04.816797 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:04.816765    5704 patch_prober.go:29] interesting pod/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused" start-of-body=
Oct 30 19:42:04.816999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:04.816826    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6" podUID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.156:15021/healthz/ready\": dial tcp 10.134.0.156:15021: connect: connection refused"
Oct 30 19:42:04.911058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:04.901198    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:05.822662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:05.822634    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:42:05.823082 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:05.823065    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:42:05.823707 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:05.823686    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:42:09.617817 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:09.617726    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:12.887562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:12.887531    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:14.504287 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:14.504247    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:15.217690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:15.217660    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:19.287999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:19.287960    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:23.999765 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:23.999728    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:28.702453 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:28.702402    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:31.143708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:31.138676    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:33.290170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:33.290084    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:33.383987 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:42:33.383953    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:42:34.451877 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:34.451845    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:42:37.885815 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:37.885782    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:42.621068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:42.618353    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:47.531616 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:47.531579    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:47.771088 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:47.771059    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:52.208527 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:52.208490    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:54.597187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:54.597156    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"]
Oct 30 19:42:54.597712 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:54.597361    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m" podUID="8758529a-cf50-4842-9124-4a8ce8499bf4" containerName="istio-proxy" containerID="cri-o://8536cb2f66ba1d447d36743af9ca5f6e0847373f5561eb0d0f97bafd91cebeea" gracePeriod=30
Oct 30 19:42:56.917051 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:56.917002    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:58.868258 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:58.868221    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:42:59.859925 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.859859    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:42:59.957961 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.957934    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-podinfo\") pod \"8758529a-cf50-4842-9124-4a8ce8499bf4\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") "
Oct 30 19:42:59.958456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.957970    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-credential-socket\") pod \"8758529a-cf50-4842-9124-4a8ce8499bf4\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") "
Oct 30 19:42:59.958456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.958000    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-data\") pod \"8758529a-cf50-4842-9124-4a8ce8499bf4\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") "
Oct 30 19:42:59.958456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.958144    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-kube-api-access-55wnw\" (UniqueName: \"kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-kube-api-access-55wnw\") pod \"8758529a-cf50-4842-9124-4a8ce8499bf4\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") "
Oct 30 19:42:59.958456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.958189    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-envoy\") pod \"8758529a-cf50-4842-9124-4a8ce8499bf4\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") "
Oct 30 19:42:59.958456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.958254    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-token\" (UniqueName: \"kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-token\") pod \"8758529a-cf50-4842-9124-4a8ce8499bf4\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") "
Oct 30 19:42:59.958456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.958282    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-credential-socket" pod "8758529a-cf50-4842-9124-4a8ce8499bf4" (UID: "8758529a-cf50-4842-9124-4a8ce8499bf4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:59.958456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.958372    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-data" pod "8758529a-cf50-4842-9124-4a8ce8499bf4" (UID: "8758529a-cf50-4842-9124-4a8ce8499bf4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:59.958456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.958415    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-socket\") pod \"8758529a-cf50-4842-9124-4a8ce8499bf4\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") "
Oct 30 19:42:59.958456 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.958449    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8758529a-cf50-4842-9124-4a8ce8499bf4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8758529a-cf50-4842-9124-4a8ce8499bf4-istiod-ca-cert\") pod \"8758529a-cf50-4842-9124-4a8ce8499bf4\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") "
Oct 30 19:42:59.959026 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.958554    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-certs\") pod \"8758529a-cf50-4842-9124-4a8ce8499bf4\" (UID: \"8758529a-cf50-4842-9124-4a8ce8499bf4\") "
Oct 30 19:42:59.959026 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.958708    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-socket" pod "8758529a-cf50-4842-9124-4a8ce8499bf4" (UID: "8758529a-cf50-4842-9124-4a8ce8499bf4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:59.959026 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.958829    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8758529a-cf50-4842-9124-4a8ce8499bf4-istiod-ca-cert" pod "8758529a-cf50-4842-9124-4a8ce8499bf4" (UID: "8758529a-cf50-4842-9124-4a8ce8499bf4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:42:59.959026 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.958902    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-certs" pod "8758529a-cf50-4842-9124-4a8ce8499bf4" (UID: "8758529a-cf50-4842-9124-4a8ce8499bf4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:59.960627 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.960603    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:59.960760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.960631    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8758529a-cf50-4842-9124-4a8ce8499bf4-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:59.960760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.960646    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:59.960760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.960660    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:59.960760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.960679    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:42:59.971488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.971288    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-kube-api-access-55wnw" pod "8758529a-cf50-4842-9124-4a8ce8499bf4" (UID: "8758529a-cf50-4842-9124-4a8ce8499bf4"). InnerVolumeSpecName "kube-api-access-55wnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:59.971857 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.971826    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-token" pod "8758529a-cf50-4842-9124-4a8ce8499bf4" (UID: "8758529a-cf50-4842-9124-4a8ce8499bf4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:42:59.971857 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.971827    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-envoy" pod "8758529a-cf50-4842-9124-4a8ce8499bf4" (UID: "8758529a-cf50-4842-9124-4a8ce8499bf4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:42:59.972198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:42:59.972183    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-podinfo" pod "8758529a-cf50-4842-9124-4a8ce8499bf4" (UID: "8758529a-cf50-4842-9124-4a8ce8499bf4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:00.061874 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.061843    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55wnw\" (UniqueName: \"kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-kube-api-access-55wnw\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:00.061874 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.061870    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:00.061874 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.061880    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:00.062130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.061888    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8758529a-cf50-4842-9124-4a8ce8499bf4-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:00.403786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.403753    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"]
Oct 30 19:43:00.404063 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.403994    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5" podUID="3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" containerName="istio-proxy" containerID="cri-o://3bcf2ffac38f0be5395c0663ed3917fa508f174ea1016d7145df6a81ba2ba4dc" gracePeriod=30
Oct 30 19:43:00.689617 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.689536    5704 generic.go:356] "Generic (PLEG): container finished" podID="8758529a-cf50-4842-9124-4a8ce8499bf4" containerID="8536cb2f66ba1d447d36743af9ca5f6e0847373f5561eb0d0f97bafd91cebeea" exitCode=0
Oct 30 19:43:00.689617 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.689592    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"
Oct 30 19:43:00.689813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.689621    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m" event={"ID":"8758529a-cf50-4842-9124-4a8ce8499bf4","Type":"ContainerDied","Data":"8536cb2f66ba1d447d36743af9ca5f6e0847373f5561eb0d0f97bafd91cebeea"}
Oct 30 19:43:00.689813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.689676    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m" event={"ID":"8758529a-cf50-4842-9124-4a8ce8499bf4","Type":"ContainerDied","Data":"3bb393d92c95d765f3af52cb47f022823fee76320d4dc8e233ed23cebc9a9b58"}
Oct 30 19:43:00.689813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.689702    5704 scope.go:122] "RemoveContainer" containerID="8536cb2f66ba1d447d36743af9ca5f6e0847373f5561eb0d0f97bafd91cebeea"
Oct 30 19:43:00.700827 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.700809    5704 scope.go:122] "RemoveContainer" containerID="8536cb2f66ba1d447d36743af9ca5f6e0847373f5561eb0d0f97bafd91cebeea"
Oct 30 19:43:00.701107 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:43:00.701086    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8536cb2f66ba1d447d36743af9ca5f6e0847373f5561eb0d0f97bafd91cebeea\": container with ID starting with 8536cb2f66ba1d447d36743af9ca5f6e0847373f5561eb0d0f97bafd91cebeea not found: ID does not exist" containerID="8536cb2f66ba1d447d36743af9ca5f6e0847373f5561eb0d0f97bafd91cebeea"
Oct 30 19:43:00.701164 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.701115    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8536cb2f66ba1d447d36743af9ca5f6e0847373f5561eb0d0f97bafd91cebeea"} err="failed to get container status \"8536cb2f66ba1d447d36743af9ca5f6e0847373f5561eb0d0f97bafd91cebeea\": rpc error: code = NotFound desc = could not find container \"8536cb2f66ba1d447d36743af9ca5f6e0847373f5561eb0d0f97bafd91cebeea\": container with ID starting with 8536cb2f66ba1d447d36743af9ca5f6e0847373f5561eb0d0f97bafd91cebeea not found: ID does not exist"
Oct 30 19:43:00.846176 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.846148    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"]
Oct 30 19:43:00.851150 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:00.851127    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-l9t2m"]
Oct 30 19:43:01.179924 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:01.179893    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8758529a-cf50-4842-9124-4a8ce8499bf4" path="/var/lib/kubelet/pods/8758529a-cf50-4842-9124-4a8ce8499bf4/volumes"
Oct 30 19:43:01.572718 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:01.572680    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:03.383545 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:43:03.383515    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:43:05.689397 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.689344    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:43:05.722613 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.722585    5704 generic.go:356] "Generic (PLEG): container finished" podID="3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" containerID="3bcf2ffac38f0be5395c0663ed3917fa508f174ea1016d7145df6a81ba2ba4dc" exitCode=0
Oct 30 19:43:05.722784 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.722643    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"
Oct 30 19:43:05.722784 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.722675    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5" event={"ID":"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6","Type":"ContainerDied","Data":"3bcf2ffac38f0be5395c0663ed3917fa508f174ea1016d7145df6a81ba2ba4dc"}
Oct 30 19:43:05.722784 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.722754    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5" event={"ID":"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6","Type":"ContainerDied","Data":"3587451a8882cfc09236de27f9e598aa46f84d630c338ae95ce901c0f47c1164"}
Oct 30 19:43:05.722784 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.722778    5704 scope.go:122] "RemoveContainer" containerID="3bcf2ffac38f0be5395c0663ed3917fa508f174ea1016d7145df6a81ba2ba4dc"
Oct 30 19:43:05.732155 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732134    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-podinfo\") pod \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") "
Oct 30 19:43:05.732265 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732172    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-credential-socket\") pod \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") "
Oct 30 19:43:05.732265 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732206    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-envoy\") pod \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") "
Oct 30 19:43:05.732265 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732234    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-certs\") pod \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") "
Oct 30 19:43:05.732427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732315    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-data\") pod \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") "
Oct 30 19:43:05.732427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732362    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-token\" (UniqueName: \"kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-token\") pod \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") "
Oct 30 19:43:05.732427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732390    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istiod-ca-cert\") pod \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") "
Oct 30 19:43:05.732427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732412    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-credential-socket" pod "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" (UID: "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.732933 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732514    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-kube-api-access-m9ltl\" (UniqueName: \"kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-kube-api-access-m9ltl\") pod \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") "
Oct 30 19:43:05.732933 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732603    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-certs" pod "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" (UID: "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.732933 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732633    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-data" pod "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" (UID: "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.732933 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732643    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-socket\") pod \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\" (UID: \"3ec3ec9b-9c71-4b27-91ef-0bab48eafad6\") "
Oct 30 19:43:05.732933 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732828    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istiod-ca-cert" pod "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" (UID: "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:43:05.733334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.732954    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-socket" pod "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" (UID: "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.733334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.733019    5704 scope.go:122] "RemoveContainer" containerID="3bcf2ffac38f0be5395c0663ed3917fa508f174ea1016d7145df6a81ba2ba4dc"
Oct 30 19:43:05.733334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.733300    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.733334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.733324    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.733553 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.733339    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.733553 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.733353    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.733553 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.733365    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.733553 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:43:05.733512    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bcf2ffac38f0be5395c0663ed3917fa508f174ea1016d7145df6a81ba2ba4dc\": container with ID starting with 3bcf2ffac38f0be5395c0663ed3917fa508f174ea1016d7145df6a81ba2ba4dc not found: ID does not exist" containerID="3bcf2ffac38f0be5395c0663ed3917fa508f174ea1016d7145df6a81ba2ba4dc"
Oct 30 19:43:05.733553 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.733539    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bcf2ffac38f0be5395c0663ed3917fa508f174ea1016d7145df6a81ba2ba4dc"} err="failed to get container status \"3bcf2ffac38f0be5395c0663ed3917fa508f174ea1016d7145df6a81ba2ba4dc\": rpc error: code = NotFound desc = could not find container \"3bcf2ffac38f0be5395c0663ed3917fa508f174ea1016d7145df6a81ba2ba4dc\": container with ID starting with 3bcf2ffac38f0be5395c0663ed3917fa508f174ea1016d7145df6a81ba2ba4dc not found: ID does not exist"
Oct 30 19:43:05.734589 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.734565    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-token" pod "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" (UID: "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:05.734702 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.734617    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-envoy" pod "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" (UID: "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:43:05.735161 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.735141    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-podinfo" pod "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" (UID: "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:43:05.735242 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.735158    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-kube-api-access-m9ltl" pod "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" (UID: "3ec3ec9b-9c71-4b27-91ef-0bab48eafad6"). InnerVolumeSpecName "kube-api-access-m9ltl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:43:05.834590 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.834560    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.834590 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.834588    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.834590 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.834597    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:05.834808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:05.834607    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m9ltl\" (UniqueName: \"kubernetes.io/projected/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6-kube-api-access-m9ltl\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:43:06.138937 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:06.138898    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"]
Oct 30 19:43:06.141891 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:06.141862    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-cz6x5"]
Oct 30 19:43:06.200868 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:06.200731    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:06.453620 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:06.453532    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:07.180515 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:07.180485    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" path="/var/lib/kubelet/pods/3ec3ec9b-9c71-4b27-91ef-0bab48eafad6/volumes"
Oct 30 19:43:10.877153 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:10.877111    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:11.967786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:11.967747    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:43:13.222557 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:13.222521    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:15.466342 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:15.466313    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:20.200058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:20.199439    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:24.773057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:24.772898    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:29.658294 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:29.653336    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:30.912518 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:30.912485    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:33.384207 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:43:33.384175    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:43:34.283071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:34.283017    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:39.131744 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:39.131699    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:43.896733 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:43.896684    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:48.731287 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:48.731250    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:48.930184 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:48.930151    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:51.749469 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:51.747288    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:53.419118 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:53.419082    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:43:58.288831 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:43:58.288792    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:01.374273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:01.374240    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:01.655982 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:01.655901    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:02.923306 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:02.923269    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:03.382950 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:44:03.382915    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:44:07.744531 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:07.744494    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:12.612272 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:12.612232    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:13.568670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:13.568638    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:14.963999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:14.963966    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:44:15.564996 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:15.564887    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:17.393215 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:17.393176    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:22.071403 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:22.071321    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:26.694132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:26.694088    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:29.464210 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:29.464166    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:31.523154 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:31.523122    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:33.383333 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:44:33.383293    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:44:36.092332 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:36.092297    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:40.834728 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:40.834688    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:45.509850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:45.509805    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:50.201088 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:50.201050    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:54.845988 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:54.845940    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:57.456672 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:57.456641    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:44:58.749068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:58.746234    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:59.390835 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:59.390798    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:44:59.606445 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:44:59.598660    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:00.298591 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.298555    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"]
Oct 30 19:45:00.299642 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.299593    5704 cpu_manager.go:414] "containerMap: removing container" podUID="8758529a-cf50-4842-9124-4a8ce8499bf4" containerName="istio-proxy"
Oct 30 19:45:00.299642 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.299641    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8758529a-cf50-4842-9124-4a8ce8499bf4" containerName="istio-proxy"
Oct 30 19:45:00.299873 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.299666    5704 cpu_manager.go:414] "containerMap: removing container" podUID="3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" containerName="istio-proxy"
Oct 30 19:45:00.299873 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.299675    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" containerName="istio-proxy"
Oct 30 19:45:00.299873 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.299857    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ec3ec9b-9c71-4b27-91ef-0bab48eafad6" containerName="istio-proxy"
Oct 30 19:45:00.300000 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.299889    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="8758529a-cf50-4842-9124-4a8ce8499bf4" containerName="istio-proxy"
Oct 30 19:45:00.304313 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.304294    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"
Oct 30 19:45:00.323694 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.323668    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzl6x\" (UniqueName: \"kubernetes.io/projected/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-kube-api-access-vzl6x\") pod \"maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg\" (UID: \"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"
Oct 30 19:45:00.323864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.323830    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg\" (UID: \"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"
Oct 30 19:45:00.353970 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.353936    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b8jtx\""
Oct 30 19:45:00.405114 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.405087    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"]
Oct 30 19:45:00.425378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.425345    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzl6x\" (UniqueName: \"kubernetes.io/projected/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-kube-api-access-vzl6x\") pod \"maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg\" (UID: \"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"
Oct 30 19:45:00.425554 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.425494    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg\" (UID: \"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"
Oct 30 19:45:00.426089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.426070    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg\" (UID: \"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"
Oct 30 19:45:00.488819 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.488785    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzl6x\" (UniqueName: \"kubernetes.io/projected/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-kube-api-access-vzl6x\") pod \"maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg\" (UID: \"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"
Oct 30 19:45:00.586128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.586043    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:00.620717 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.620685    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"
Oct 30 19:45:00.808121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.808071    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:00.992234 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.992206    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"]
Oct 30 19:45:00.994050 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:45:00.994008    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c2e2ea6_1fb2_4bfd_80a5_425dcd4b7622.slice/crio-dbc0ca925f99a9a8c0c27fd6930a9061c105fe7dc0be0911fb5876379ff913e1 WatchSource:0}: Error finding container dbc0ca925f99a9a8c0c27fd6930a9061c105fe7dc0be0911fb5876379ff913e1: Status 404 returned error can't find the container with id dbc0ca925f99a9a8c0c27fd6930a9061c105fe7dc0be0911fb5876379ff913e1
Oct 30 19:45:00.996352 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:00.996333    5704 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:45:01.362608 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:01.362580    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"]
Oct 30 19:45:01.363122 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:01.362772    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj" podUID="c46c3c63-cf65-47cd-addc-41b0db2f23ae" containerName="istio-proxy" containerID="cri-o://7fabc7ffefe0bae99ae62cd820f445b8e8d2e065060420d1abd444fe97f4fd86" gracePeriod=30
Oct 30 19:45:01.596886 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:01.596842    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg" event={"ID":"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622","Type":"ContainerStarted","Data":"dbc0ca925f99a9a8c0c27fd6930a9061c105fe7dc0be0911fb5876379ff913e1"}
Oct 30 19:45:03.385162 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:45:03.385127    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:45:03.521019 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:03.520979    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:04.590699 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:04.590664    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:05.626206 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:05.626129    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg" event={"ID":"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622","Type":"ContainerStarted","Data":"ad2115e4126db111a468d981746327d87f88483095e054c85f8067f863c52975"}
Oct 30 19:45:05.749304 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:05.749253    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg" podStartSLOduration=1.686530023 podStartE2EDuration="5.749238991s" totalImagesPullingTime="4.062708968s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:05.747233388 +0000 UTC m=+3478.693401292" watchObservedRunningTime="2026-10-30 19:45:05.749238991 +0000 UTC m=+3478.695406905"
Oct 30 19:45:05.829833 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:05.829795    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:06.115939 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.115908    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:06.623755 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.623732    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:45:06.633885 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.633823    5704 generic.go:356] "Generic (PLEG): container finished" podID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" containerID="ad2115e4126db111a468d981746327d87f88483095e054c85f8067f863c52975" exitCode=60
Oct 30 19:45:06.634292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.633901    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg" event={"ID":"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622","Type":"ContainerDied","Data":"ad2115e4126db111a468d981746327d87f88483095e054c85f8067f863c52975"}
Oct 30 19:45:06.634292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.634127    5704 scope.go:122] "RemoveContainer" containerID="ad2115e4126db111a468d981746327d87f88483095e054c85f8067f863c52975"
Oct 30 19:45:06.635485 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.635463    5704 generic.go:356] "Generic (PLEG): container finished" podID="c46c3c63-cf65-47cd-addc-41b0db2f23ae" containerID="7fabc7ffefe0bae99ae62cd820f445b8e8d2e065060420d1abd444fe97f4fd86" exitCode=0
Oct 30 19:45:06.635580 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.635509    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"
Oct 30 19:45:06.635642 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.635503    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj" event={"ID":"c46c3c63-cf65-47cd-addc-41b0db2f23ae","Type":"ContainerDied","Data":"7fabc7ffefe0bae99ae62cd820f445b8e8d2e065060420d1abd444fe97f4fd86"}
Oct 30 19:45:06.635642 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.635622    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj" event={"ID":"c46c3c63-cf65-47cd-addc-41b0db2f23ae","Type":"ContainerDied","Data":"5c17e2a809c0f7ae8b9c4d1de117b0614776696996578dcb12f20ee6d8d6e576"}
Oct 30 19:45:06.635726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.635647    5704 scope.go:122] "RemoveContainer" containerID="7fabc7ffefe0bae99ae62cd820f445b8e8d2e065060420d1abd444fe97f4fd86"
Oct 30 19:45:06.649515 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.649498    5704 scope.go:122] "RemoveContainer" containerID="7fabc7ffefe0bae99ae62cd820f445b8e8d2e065060420d1abd444fe97f4fd86"
Oct 30 19:45:06.649793 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:45:06.649764    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fabc7ffefe0bae99ae62cd820f445b8e8d2e065060420d1abd444fe97f4fd86\": container with ID starting with 7fabc7ffefe0bae99ae62cd820f445b8e8d2e065060420d1abd444fe97f4fd86 not found: ID does not exist" containerID="7fabc7ffefe0bae99ae62cd820f445b8e8d2e065060420d1abd444fe97f4fd86"
Oct 30 19:45:06.649903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.649801    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fabc7ffefe0bae99ae62cd820f445b8e8d2e065060420d1abd444fe97f4fd86"} err="failed to get container status \"7fabc7ffefe0bae99ae62cd820f445b8e8d2e065060420d1abd444fe97f4fd86\": rpc error: code = NotFound desc = could not find container \"7fabc7ffefe0bae99ae62cd820f445b8e8d2e065060420d1abd444fe97f4fd86\": container with ID starting with 7fabc7ffefe0bae99ae62cd820f445b8e8d2e065060420d1abd444fe97f4fd86 not found: ID does not exist"
Oct 30 19:45:06.704103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704079    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-envoy\") pod \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") "
Oct 30 19:45:06.704227 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704150    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-token\" (UniqueName: \"kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-token\") pod \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") "
Oct 30 19:45:06.704280 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704232    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-data\") pod \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") "
Oct 30 19:45:06.704341 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704287    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-credential-socket\") pod \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") "
Oct 30 19:45:06.704402 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704381    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-socket\") pod \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") "
Oct 30 19:45:06.704461 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704427    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-kube-api-access-9vzwr\" (UniqueName: \"kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-kube-api-access-9vzwr\") pod \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") "
Oct 30 19:45:06.704521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704489    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-podinfo\") pod \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") "
Oct 30 19:45:06.704521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704512    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istiod-ca-cert\") pod \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") "
Oct 30 19:45:06.704625 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704547    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-certs\") pod \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\" (UID: \"c46c3c63-cf65-47cd-addc-41b0db2f23ae\") "
Oct 30 19:45:06.704625 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704558    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-data" pod "c46c3c63-cf65-47cd-addc-41b0db2f23ae" (UID: "c46c3c63-cf65-47cd-addc-41b0db2f23ae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:06.704727 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704600    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-credential-socket" pod "c46c3c63-cf65-47cd-addc-41b0db2f23ae" (UID: "c46c3c63-cf65-47cd-addc-41b0db2f23ae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:06.704727 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704639    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-socket" pod "c46c3c63-cf65-47cd-addc-41b0db2f23ae" (UID: "c46c3c63-cf65-47cd-addc-41b0db2f23ae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:06.704830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704764    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-certs" pod "c46c3c63-cf65-47cd-addc-41b0db2f23ae" (UID: "c46c3c63-cf65-47cd-addc-41b0db2f23ae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:06.704948 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.704925    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istiod-ca-cert" pod "c46c3c63-cf65-47cd-addc-41b0db2f23ae" (UID: "c46c3c63-cf65-47cd-addc-41b0db2f23ae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:06.705301 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.705278    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:06.705404 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.705307    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:06.705404 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.705324    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:06.705404 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.705339    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:06.705404 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.705354    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:06.706421 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.706394    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-token" pod "c46c3c63-cf65-47cd-addc-41b0db2f23ae" (UID: "c46c3c63-cf65-47cd-addc-41b0db2f23ae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:06.706610 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.706514    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-envoy" pod "c46c3c63-cf65-47cd-addc-41b0db2f23ae" (UID: "c46c3c63-cf65-47cd-addc-41b0db2f23ae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:45:06.706696 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.706676    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-podinfo" pod "c46c3c63-cf65-47cd-addc-41b0db2f23ae" (UID: "c46c3c63-cf65-47cd-addc-41b0db2f23ae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:45:06.706833 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.706816    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-kube-api-access-9vzwr" pod "c46c3c63-cf65-47cd-addc-41b0db2f23ae" (UID: "c46c3c63-cf65-47cd-addc-41b0db2f23ae"). InnerVolumeSpecName "kube-api-access-9vzwr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:06.806742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.806714    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:06.806742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.806741    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:06.806973 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.806754    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:06.806973 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:06.806766    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vzwr\" (UniqueName: \"kubernetes.io/projected/c46c3c63-cf65-47cd-addc-41b0db2f23ae-kube-api-access-9vzwr\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:07.114239 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:07.114206    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"]
Oct 30 19:45:07.130002 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:07.129974    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-2301cb-openshift-default-7999cbc45f-djrcj"]
Oct 30 19:45:07.180745 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:07.180714    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c46c3c63-cf65-47cd-addc-41b0db2f23ae" path="/var/lib/kubelet/pods/c46c3c63-cf65-47cd-addc-41b0db2f23ae/volumes"
Oct 30 19:45:07.644115 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:07.644080    5704 generic.go:356] "Generic (PLEG): container finished" podID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" containerID="2bf1d3772df2acddcf8cc9cfc48ccfa268f8c7ee7d837bf53fdc4b571e9198e2" exitCode=60
Oct 30 19:45:07.644489 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:07.644170    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg" event={"ID":"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622","Type":"ContainerDied","Data":"2bf1d3772df2acddcf8cc9cfc48ccfa268f8c7ee7d837bf53fdc4b571e9198e2"}
Oct 30 19:45:07.644489 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:07.644257    5704 scope.go:122] "RemoveContainer" containerID="ad2115e4126db111a468d981746327d87f88483095e054c85f8067f863c52975"
Oct 30 19:45:07.644489 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:07.644474    5704 scope.go:122] "RemoveContainer" containerID="2bf1d3772df2acddcf8cc9cfc48ccfa268f8c7ee7d837bf53fdc4b571e9198e2"
Oct 30 19:45:07.644761 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:45:07.644738    5704 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg_odh-ai-gateway-infra(4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg" podUID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622"
Oct 30 19:45:08.651950 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:08.651919    5704 scope.go:122] "RemoveContainer" containerID="2bf1d3772df2acddcf8cc9cfc48ccfa268f8c7ee7d837bf53fdc4b571e9198e2"
Oct 30 19:45:08.652420 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:45:08.652212    5704 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg_odh-ai-gateway-infra(4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg" podUID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622"
Oct 30 19:45:09.099751 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.099721    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:09.317680 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.317644    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:09.458989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.458915    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"]
Oct 30 19:45:09.459603 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.459587    5704 cpu_manager.go:414] "containerMap: removing container" podUID="c46c3c63-cf65-47cd-addc-41b0db2f23ae" containerName="istio-proxy"
Oct 30 19:45:09.459652 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.459605    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c46c3c63-cf65-47cd-addc-41b0db2f23ae" containerName="istio-proxy"
Oct 30 19:45:09.459743 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.459734    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="c46c3c63-cf65-47cd-addc-41b0db2f23ae" containerName="istio-proxy"
Oct 30 19:45:09.465096 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.465075    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"
Oct 30 19:45:09.515569 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.515527    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-f858d4-dockercfg-9jvrk\""
Oct 30 19:45:09.516049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.516014    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-f858d4\""
Oct 30 19:45:09.558072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.552890    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab659c59-028b-48cb-9510-bdd55e3907fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv\" (UID: \"ab659c59-028b-48cb-9510-bdd55e3907fe\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"
Oct 30 19:45:09.558072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.553095    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxtmk\" (UniqueName: \"kubernetes.io/projected/ab659c59-028b-48cb-9510-bdd55e3907fe-kube-api-access-kxtmk\") pod \"payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv\" (UID: \"ab659c59-028b-48cb-9510-bdd55e3907fe\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"
Oct 30 19:45:09.598863 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.598801    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"]
Oct 30 19:45:09.654200 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.654167    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxtmk\" (UniqueName: \"kubernetes.io/projected/ab659c59-028b-48cb-9510-bdd55e3907fe-kube-api-access-kxtmk\") pod \"payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv\" (UID: \"ab659c59-028b-48cb-9510-bdd55e3907fe\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"
Oct 30 19:45:09.654620 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.654281    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab659c59-028b-48cb-9510-bdd55e3907fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv\" (UID: \"ab659c59-028b-48cb-9510-bdd55e3907fe\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"
Oct 30 19:45:09.655192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.655163    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab659c59-028b-48cb-9510-bdd55e3907fe-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv\" (UID: \"ab659c59-028b-48cb-9510-bdd55e3907fe\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"
Oct 30 19:45:09.683442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.683416    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"]
Oct 30 19:45:09.687570 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.687552    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"
Oct 30 19:45:09.737163 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.737095    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxtmk\" (UniqueName: \"kubernetes.io/projected/ab659c59-028b-48cb-9510-bdd55e3907fe-kube-api-access-kxtmk\") pod \"payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv\" (UID: \"ab659c59-028b-48cb-9510-bdd55e3907fe\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"
Oct 30 19:45:09.755632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.755599    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e05c1aac-1693-47c7-9a11-cfa2249d3635-plugins-config-volume\") pod \"payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw\" (UID: \"e05c1aac-1693-47c7-9a11-cfa2249d3635\") " pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"
Oct 30 19:45:09.755772 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.755641    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crq85\" (UniqueName: \"kubernetes.io/projected/e05c1aac-1693-47c7-9a11-cfa2249d3635-kube-api-access-crq85\") pod \"payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw\" (UID: \"e05c1aac-1693-47c7-9a11-cfa2249d3635\") " pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"
Oct 30 19:45:09.765741 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.765708    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:09.767622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.767600    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"]
Oct 30 19:45:09.779065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.779025    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"
Oct 30 19:45:09.859093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.858794    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e05c1aac-1693-47c7-9a11-cfa2249d3635-plugins-config-volume\") pod \"payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw\" (UID: \"e05c1aac-1693-47c7-9a11-cfa2249d3635\") " pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"
Oct 30 19:45:09.859093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.858863    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crq85\" (UniqueName: \"kubernetes.io/projected/e05c1aac-1693-47c7-9a11-cfa2249d3635-kube-api-access-crq85\") pod \"payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw\" (UID: \"e05c1aac-1693-47c7-9a11-cfa2249d3635\") " pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"
Oct 30 19:45:09.859485 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.859466    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e05c1aac-1693-47c7-9a11-cfa2249d3635-plugins-config-volume\") pod \"payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw\" (UID: \"e05c1aac-1693-47c7-9a11-cfa2249d3635\") " pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"
Oct 30 19:45:09.917354 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:09.917318    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-crq85\" (UniqueName: \"kubernetes.io/projected/e05c1aac-1693-47c7-9a11-cfa2249d3635-kube-api-access-crq85\") pod \"payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw\" (UID: \"e05c1aac-1693-47c7-9a11-cfa2249d3635\") " pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"
Oct 30 19:45:10.015491 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:10.015458    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"
Oct 30 19:45:10.169006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:10.168983    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"]
Oct 30 19:45:10.171119 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:45:10.171090    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab659c59_028b_48cb_9510_bdd55e3907fe.slice/crio-ac33ad6a31304edcd12d936adc5e5c1ef390c8e8994becc29cb13ade43bdfa4a WatchSource:0}: Error finding container ac33ad6a31304edcd12d936adc5e5c1ef390c8e8994becc29cb13ade43bdfa4a: Status 404 returned error can't find the container with id ac33ad6a31304edcd12d936adc5e5c1ef390c8e8994becc29cb13ade43bdfa4a
Oct 30 19:45:10.398442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:10.398418    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"]
Oct 30 19:45:10.400281 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:45:10.400255    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode05c1aac_1693_47c7_9a11_cfa2249d3635.slice/crio-410cb40797d9ef42c0a2730de487eefaea161698a4d3ca8d2931d48510aef59b WatchSource:0}: Error finding container 410cb40797d9ef42c0a2730de487eefaea161698a4d3ca8d2931d48510aef59b: Status 404 returned error can't find the container with id 410cb40797d9ef42c0a2730de487eefaea161698a4d3ca8d2931d48510aef59b
Oct 30 19:45:10.666773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:10.666683    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw" event={"ID":"e05c1aac-1693-47c7-9a11-cfa2249d3635","Type":"ContainerStarted","Data":"49de41a7d751293e3d56a495d9e92207f58bacf5f3722cee67432f27a95874db"}
Oct 30 19:45:10.666773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:10.666751    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"
Oct 30 19:45:10.667311 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:10.666780    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw" event={"ID":"e05c1aac-1693-47c7-9a11-cfa2249d3635","Type":"ContainerStarted","Data":"410cb40797d9ef42c0a2730de487eefaea161698a4d3ca8d2931d48510aef59b"}
Oct 30 19:45:10.668462 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:10.668421    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv" event={"ID":"ab659c59-028b-48cb-9510-bdd55e3907fe","Type":"ContainerStarted","Data":"448bc8e22f40179e377cdb66a1ef7662c55b292b48d90162d1a94b458da01c25"}
Oct 30 19:45:10.668609 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:10.668479    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"
Oct 30 19:45:10.668609 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:10.668504    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv" event={"ID":"ab659c59-028b-48cb-9510-bdd55e3907fe","Type":"ContainerStarted","Data":"ac33ad6a31304edcd12d936adc5e5c1ef390c8e8994becc29cb13ade43bdfa4a"}
Oct 30 19:45:10.779612 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:10.779563    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw" podStartSLOduration=1.779548415 podStartE2EDuration="1.779548415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:10.779449393 +0000 UTC m=+3483.725617297" watchObservedRunningTime="2026-10-30 19:45:10.779548415 +0000 UTC m=+3483.725716317"
Oct 30 19:45:10.893652 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:10.893596    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv" podStartSLOduration=1.893581994 podStartE2EDuration="1.893581994s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:45:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:45:10.892863193 +0000 UTC m=+3483.839031078" watchObservedRunningTime="2026-10-30 19:45:10.893581994 +0000 UTC m=+3483.839749897"
Oct 30 19:45:13.890525 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:13.890489    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:14.498895 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:14.498849    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:19.565617 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:19.565579    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:21.678539 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:21.678514    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"
Oct 30 19:45:21.679011 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:21.678582    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"
Oct 30 19:45:22.173472 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:22.172437    5704 scope.go:122] "RemoveContainer" containerID="2bf1d3772df2acddcf8cc9cfc48ccfa268f8c7ee7d837bf53fdc4b571e9198e2"
Oct 30 19:45:22.746833 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:22.746798    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg" event={"ID":"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622","Type":"ContainerStarted","Data":"7985dad20630fa4a79d6265e14f5a94c99868d4b4a4a0731fc03690a5d60407e"}
Oct 30 19:45:23.755672 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:23.755642    5704 generic.go:356] "Generic (PLEG): container finished" podID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" containerID="7985dad20630fa4a79d6265e14f5a94c99868d4b4a4a0731fc03690a5d60407e" exitCode=60
Oct 30 19:45:23.756168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:23.755718    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg" event={"ID":"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622","Type":"ContainerDied","Data":"7985dad20630fa4a79d6265e14f5a94c99868d4b4a4a0731fc03690a5d60407e"}
Oct 30 19:45:23.756168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:23.755786    5704 scope.go:122] "RemoveContainer" containerID="2bf1d3772df2acddcf8cc9cfc48ccfa268f8c7ee7d837bf53fdc4b571e9198e2"
Oct 30 19:45:23.756168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:23.756136    5704 scope.go:122] "RemoveContainer" containerID="7985dad20630fa4a79d6265e14f5a94c99868d4b4a4a0731fc03690a5d60407e"
Oct 30 19:45:23.756438 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:45:23.756417    5704 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg_odh-ai-gateway-infra(4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg" podUID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622"
Oct 30 19:45:23.886918 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:23.886879    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"]
Oct 30 19:45:24.343728 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:24.343184    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:24.915762 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:24.915736    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"
Oct 30 19:45:24.992844 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:24.992808    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-maas-api-service-ca\") pod \"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622\" (UID: \"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622\") "
Oct 30 19:45:24.993000 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:24.992900    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-kube-api-access-vzl6x\" (UniqueName: \"kubernetes.io/projected/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-kube-api-access-vzl6x\") pod \"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622\" (UID: \"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622\") "
Oct 30 19:45:24.993300 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:24.993273    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-maas-api-service-ca" pod "4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" (UID: "4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:45:24.993528 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:24.993500    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-maas-api-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:24.995450 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:24.995432    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-kube-api-access-vzl6x" pod "4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" (UID: "4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622"). InnerVolumeSpecName "kube-api-access-vzl6x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:45:25.094610 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:25.094537    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzl6x\" (UniqueName: \"kubernetes.io/projected/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622-kube-api-access-vzl6x\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:45:25.771657 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:25.771625    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"
Oct 30 19:45:25.771657 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:25.771631    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg" event={"ID":"4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622","Type":"ContainerDied","Data":"dbc0ca925f99a9a8c0c27fd6930a9061c105fe7dc0be0911fb5876379ff913e1"}
Oct 30 19:45:25.771923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:25.771710    5704 scope.go:122] "RemoveContainer" containerID="7985dad20630fa4a79d6265e14f5a94c99868d4b4a4a0731fc03690a5d60407e"
Oct 30 19:45:25.928049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:25.928004    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"]
Oct 30 19:45:25.939618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:25.939593    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-330025-29889825-zsqlg"]
Oct 30 19:45:27.181934 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:27.181903    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" path="/var/lib/kubelet/pods/4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622/volumes"
Oct 30 19:45:29.228057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:29.224703    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:33.384072 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:45:33.384021    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:45:33.894121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:33.894016    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:38.468712 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:38.468668    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:43.037111 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:43.037075    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:47.712687 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:47.712653    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:52.283846 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:52.283810    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:56.989700 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:56.989643    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:58.822094 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:58.822059    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:45:59.068418 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:45:59.068380    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:00.279487 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:00.279450    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:46:01.724076 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:01.709990    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:03.384091 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:46:03.384050    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:46:06.259273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:06.259240    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:10.971780 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:10.971740    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:11.202211 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:11.202178    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"]
Oct 30 19:46:11.202410 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:11.202385    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" podUID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" containerName="istio-proxy" containerID="cri-o://c8c0d1861cd31efc44d3db5377e04690855fba0743ab72f9818a7468330de773" gracePeriod=30
Oct 30 19:46:14.211803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:14.211766    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:15.721931 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:15.721892    5704 patch_prober.go:29] interesting pod/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:46:15.722318 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:15.721969    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" podUID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:46:15.976409 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:15.976320    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:16.467843 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.467819    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:46:16.493231 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.493172    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-podinfo\") pod \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") "
Oct 30 19:46:16.493349 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.493299    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-data\") pod \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") "
Oct 30 19:46:16.493389 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.493346    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-socket\") pod \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") "
Oct 30 19:46:16.493426 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.493393    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-kube-api-access-ffd74\" (UniqueName: \"kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-kube-api-access-ffd74\") pod \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") "
Oct 30 19:46:16.493469 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.493429    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-token\" (UniqueName: \"kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-token\") pod \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") "
Oct 30 19:46:16.493540 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.493524    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istiod-ca-cert\") pod \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") "
Oct 30 19:46:16.493603 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.493574    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-data" pod "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" (UID: "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:16.493664 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.493579    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-envoy\") pod \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") "
Oct 30 19:46:16.493724 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.493658    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-socket" pod "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" (UID: "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:16.493724 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.493671    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-credential-socket\") pod \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") "
Oct 30 19:46:16.493840 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.493729    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-certs\") pod \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\" (UID: \"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6\") "
Oct 30 19:46:16.493987 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.493962    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-credential-socket" pod "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" (UID: "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:16.494087 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.493972    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istiod-ca-cert" pod "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" (UID: "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:46:16.494206 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.494188    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-certs" pod "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" (UID: "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:16.494412 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.494397    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:16.494412 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.494412    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:16.494546 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.494421    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:16.494546 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.494431    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:16.494546 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.494440    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:16.495565 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.495543    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-kube-api-access-ffd74" pod "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" (UID: "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6"). InnerVolumeSpecName "kube-api-access-ffd74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:16.495664 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.495643    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-token" pod "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" (UID: "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:46:16.495851 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.495823    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-podinfo" pod "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" (UID: "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:46:16.495851 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.495833    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-envoy" pod "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" (UID: "c1036dd5-2b14-45fb-a368-b3ef1fccc7c6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:46:16.595092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.595061    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:16.595092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.595089    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:16.595254 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.595099    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ffd74\" (UniqueName: \"kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-kube-api-access-ffd74\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:16.595254 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:16.595110    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:46:17.122247 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:17.122215    5704 generic.go:356] "Generic (PLEG): container finished" podID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" containerID="c8c0d1861cd31efc44d3db5377e04690855fba0743ab72f9818a7468330de773" exitCode=0
Oct 30 19:46:17.122677 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:17.122274    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"
Oct 30 19:46:17.122677 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:17.122293    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" event={"ID":"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6","Type":"ContainerDied","Data":"c8c0d1861cd31efc44d3db5377e04690855fba0743ab72f9818a7468330de773"}
Oct 30 19:46:17.122677 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:17.122371    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw" event={"ID":"c1036dd5-2b14-45fb-a368-b3ef1fccc7c6","Type":"ContainerDied","Data":"449e49acaa7c36468f9d7f17c5b73776084c2f1a3ce2f8fccc1f5c3e6793bb3f"}
Oct 30 19:46:17.122677 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:17.122396    5704 scope.go:122] "RemoveContainer" containerID="c8c0d1861cd31efc44d3db5377e04690855fba0743ab72f9818a7468330de773"
Oct 30 19:46:17.133511 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:17.133494    5704 scope.go:122] "RemoveContainer" containerID="c8c0d1861cd31efc44d3db5377e04690855fba0743ab72f9818a7468330de773"
Oct 30 19:46:17.133768 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:46:17.133748    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8c0d1861cd31efc44d3db5377e04690855fba0743ab72f9818a7468330de773\": container with ID starting with c8c0d1861cd31efc44d3db5377e04690855fba0743ab72f9818a7468330de773 not found: ID does not exist" containerID="c8c0d1861cd31efc44d3db5377e04690855fba0743ab72f9818a7468330de773"
Oct 30 19:46:17.133842 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:17.133777    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8c0d1861cd31efc44d3db5377e04690855fba0743ab72f9818a7468330de773"} err="failed to get container status \"c8c0d1861cd31efc44d3db5377e04690855fba0743ab72f9818a7468330de773\": rpc error: code = NotFound desc = could not find container \"c8c0d1861cd31efc44d3db5377e04690855fba0743ab72f9818a7468330de773\": container with ID starting with c8c0d1861cd31efc44d3db5377e04690855fba0743ab72f9818a7468330de773 not found: ID does not exist"
Oct 30 19:46:17.294644 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:17.294613    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"]
Oct 30 19:46:17.298849 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:17.298817    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-wfmgw"]
Oct 30 19:46:19.180120 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:19.180089    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" path="/var/lib/kubelet/pods/c1036dd5-2b14-45fb-a368-b3ef1fccc7c6/volumes"
Oct 30 19:46:20.080373 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:20.080335    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:20.309066 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:20.309011    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:20.489097 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:20.488996    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:24.840310 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:24.840271    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:29.576783 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:29.576743    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:33.383137 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:46:33.383100    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:46:34.487898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:34.487858    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:39.067065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:39.066179    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:40.636605 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:40.636570    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"]
Oct 30 19:46:41.018537 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:41.018507    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:41.206591 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:41.206546    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:43.669927 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:43.669893    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:45.637538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:45.637480    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp" podUID="4ea5bfd4-5c12-462d-beb8-2944091571f5" containerName="payload-processing" containerID="cri-o://a481592aba4ca7c223d9bfd09a9775dba48225fda0d098536c9a49c8fbdb8827" gracePeriod=25
Oct 30 19:46:47.119413 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:47.119377    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp" podUID="4ea5bfd4-5c12-462d-beb8-2944091571f5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:48.282770 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:48.282616    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:52.879249 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:52.879211    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:54.130334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:54.130256    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:57.118689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:57.118655    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp" podUID="4ea5bfd4-5c12-462d-beb8-2944091571f5" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:46:57.503103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:57.503068    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:46:58.236727 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:46:58.236699    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:47:00.441368 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:00.441336    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:02.324606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:02.324553    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:03.383712 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:47:03.383679    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:47:05.900370 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:05.900310    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"
Oct 30 19:47:05.919786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:05.919761    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ea5bfd4-5c12-462d-beb8-2944091571f5-kube-api-access-lqzrl\" (UniqueName: \"kubernetes.io/projected/4ea5bfd4-5c12-462d-beb8-2944091571f5-kube-api-access-lqzrl\") pod \"4ea5bfd4-5c12-462d-beb8-2944091571f5\" (UID: \"4ea5bfd4-5c12-462d-beb8-2944091571f5\") "
Oct 30 19:47:05.919912 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:05.919830    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4ea5bfd4-5c12-462d-beb8-2944091571f5-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ea5bfd4-5c12-462d-beb8-2944091571f5-plugins-config-volume\") pod \"4ea5bfd4-5c12-462d-beb8-2944091571f5\" (UID: \"4ea5bfd4-5c12-462d-beb8-2944091571f5\") "
Oct 30 19:47:05.920233 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:05.920205    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ea5bfd4-5c12-462d-beb8-2944091571f5-plugins-config-volume" pod "4ea5bfd4-5c12-462d-beb8-2944091571f5" (UID: "4ea5bfd4-5c12-462d-beb8-2944091571f5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:47:05.920350 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:05.920327    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ea5bfd4-5c12-462d-beb8-2944091571f5-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:05.921775 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:05.921753    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ea5bfd4-5c12-462d-beb8-2944091571f5-kube-api-access-lqzrl" pod "4ea5bfd4-5c12-462d-beb8-2944091571f5" (UID: "4ea5bfd4-5c12-462d-beb8-2944091571f5"). InnerVolumeSpecName "kube-api-access-lqzrl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:47:06.021332 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:06.021302    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqzrl\" (UniqueName: \"kubernetes.io/projected/4ea5bfd4-5c12-462d-beb8-2944091571f5-kube-api-access-lqzrl\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:47:06.444708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:06.444680    5704 generic.go:356] "Generic (PLEG): container finished" podID="4ea5bfd4-5c12-462d-beb8-2944091571f5" containerID="a481592aba4ca7c223d9bfd09a9775dba48225fda0d098536c9a49c8fbdb8827" exitCode=0
Oct 30 19:47:06.444864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:06.444725    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"
Oct 30 19:47:06.444864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:06.444760    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp" event={"ID":"4ea5bfd4-5c12-462d-beb8-2944091571f5","Type":"ContainerDied","Data":"a481592aba4ca7c223d9bfd09a9775dba48225fda0d098536c9a49c8fbdb8827"}
Oct 30 19:47:06.444864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:06.444812    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp" event={"ID":"4ea5bfd4-5c12-462d-beb8-2944091571f5","Type":"ContainerDied","Data":"f4f28f8fa03b41afb5aec0d8bcc7353ff3f35eaf08f55238f1c1089e32d9b697"}
Oct 30 19:47:06.444864 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:06.444840    5704 scope.go:122] "RemoveContainer" containerID="a481592aba4ca7c223d9bfd09a9775dba48225fda0d098536c9a49c8fbdb8827"
Oct 30 19:47:06.455487 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:06.455473    5704 scope.go:122] "RemoveContainer" containerID="a481592aba4ca7c223d9bfd09a9775dba48225fda0d098536c9a49c8fbdb8827"
Oct 30 19:47:06.455777 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:47:06.455754    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a481592aba4ca7c223d9bfd09a9775dba48225fda0d098536c9a49c8fbdb8827\": container with ID starting with a481592aba4ca7c223d9bfd09a9775dba48225fda0d098536c9a49c8fbdb8827 not found: ID does not exist" containerID="a481592aba4ca7c223d9bfd09a9775dba48225fda0d098536c9a49c8fbdb8827"
Oct 30 19:47:06.455873 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:06.455782    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a481592aba4ca7c223d9bfd09a9775dba48225fda0d098536c9a49c8fbdb8827"} err="failed to get container status \"a481592aba4ca7c223d9bfd09a9775dba48225fda0d098536c9a49c8fbdb8827\": rpc error: code = NotFound desc = could not find container \"a481592aba4ca7c223d9bfd09a9775dba48225fda0d098536c9a49c8fbdb8827\": container with ID starting with a481592aba4ca7c223d9bfd09a9775dba48225fda0d098536c9a49c8fbdb8827 not found: ID does not exist"
Oct 30 19:47:06.599880 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:06.599854    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"]
Oct 30 19:47:06.645836 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:06.645807    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-praxis-a-af72cf-586d9db9fd-76pzp"]
Oct 30 19:47:07.006445 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:07.006313    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:07.180591 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:07.180557    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ea5bfd4-5c12-462d-beb8-2944091571f5" path="/var/lib/kubelet/pods/4ea5bfd4-5c12-462d-beb8-2944091571f5/volumes"
Oct 30 19:47:11.799715 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:11.799679    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:14.535811 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:14.535779    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:16.510540 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:16.510503    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:21.263747 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:21.263666    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:25.852335 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:25.852217    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:30.453899 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:30.453859    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:30.638574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:30.638542    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:31.819967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:31.819928    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:33.383149 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:47:33.383112    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:47:35.068743 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:35.068702    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:39.706530 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:39.706492    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:44.747786 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:44.747748    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:44.961058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:44.960996    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:45.367924 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:45.367892    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:47:49.690894 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:49.690857    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:52.753315 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:52.753281    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:53.371566 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:53.371527    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:53.577975 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:53.577940    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:54.404303 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:54.403977    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:57.738170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:57.738136    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:57.964884 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:57.964844    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:47:59.282208 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:47:59.282171    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:00.915075 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:00.915013    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:01.118762 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:01.118729    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:03.383776 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:48:03.383743    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:48:04.057799 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:04.057761    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:08.634175 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:08.634116    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:13.337055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:13.332242    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:14.874904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:14.874873    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:17.929058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:17.928562    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:22.598420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:22.598385    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:27.240333 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:27.240295    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:31.914923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:31.914810    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:33.384216 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:48:33.384182    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:48:36.469707 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:36.469664    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:39.160104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:39.160069    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:40.514500 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:40.514464    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:41.957870 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:41.957838    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:42.080491 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:42.080457    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:48:42.285262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:42.285222    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:42.482048 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:42.482010    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:42.956852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:42.956813    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:47.196459 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:47.196422    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:48.018898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:48.018819    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:51.892747 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:51.892718    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:56.483096 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:56.483055    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:48:57.895239 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:48:57.895200    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:01.115710 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:01.115668    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:03.383305 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:49:03.383269    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:49:05.719680 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:05.719642    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:06.122235 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:06.122202    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:10.399244 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:10.399200    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:14.875743 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:14.875704    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:15.101338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:15.101302    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:15.282946 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:15.282909    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:15.476925 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:15.476891    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:19.641776 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:19.641692    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:24.301239 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:24.301206    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:24.545270 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:24.543746    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:25.889693 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:25.889651    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:27.651964 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:27.651928    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:28.886389 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:28.886356    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:29.181357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:29.181270    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:29.805085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:29.805046    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:30.094930 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:30.094845    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:32.021713 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:32.021677    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:33.384290 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:49:33.384255    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:49:33.606838 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:33.606800    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:34.348289 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:34.348238    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:34.527868 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:34.527829    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:35.264871 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:35.264841    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:37.999336 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:37.999304    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:38.181591 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:38.181541    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:38.678184 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:38.678155    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:43.319619 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:43.319584    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:46.865108 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:46.865073    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"]
Oct 30 19:49:46.964020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:46.963992    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"]
Oct 30 19:49:47.244613 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:47.244520    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:47.436331 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:47.436293    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:47.995140 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:47.995112    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:49:48.101086 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:48.099310    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:51.866205 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:51.866171    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw" podUID="e05c1aac-1693-47c7-9a11-cfa2249d3635" containerName="payload-processing" containerID="cri-o://49de41a7d751293e3d56a495d9e92207f58bacf5f3722cee67432f27a95874db" gracePeriod=25
Oct 30 19:49:51.964465 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:51.964433    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv" podUID="ab659c59-028b-48cb-9510-bdd55e3907fe" containerName="payload-pre-processing" containerID="cri-o://448bc8e22f40179e377cdb66a1ef7662c55b292b48d90162d1a94b458da01c25" gracePeriod=25
Oct 30 19:49:52.125941 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.125877    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"
Oct 30 19:49:52.222380 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.222357    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e05c1aac-1693-47c7-9a11-cfa2249d3635-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e05c1aac-1693-47c7-9a11-cfa2249d3635-plugins-config-volume\") pod \"e05c1aac-1693-47c7-9a11-cfa2249d3635\" (UID: \"e05c1aac-1693-47c7-9a11-cfa2249d3635\") "
Oct 30 19:49:52.222553 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.222536    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e05c1aac-1693-47c7-9a11-cfa2249d3635-kube-api-access-crq85\" (UniqueName: \"kubernetes.io/projected/e05c1aac-1693-47c7-9a11-cfa2249d3635-kube-api-access-crq85\") pod \"e05c1aac-1693-47c7-9a11-cfa2249d3635\" (UID: \"e05c1aac-1693-47c7-9a11-cfa2249d3635\") "
Oct 30 19:49:52.222739 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.222716    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e05c1aac-1693-47c7-9a11-cfa2249d3635-plugins-config-volume" pod "e05c1aac-1693-47c7-9a11-cfa2249d3635" (UID: "e05c1aac-1693-47c7-9a11-cfa2249d3635"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:52.222964 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.222947    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e05c1aac-1693-47c7-9a11-cfa2249d3635-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:52.224621 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.224581    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e05c1aac-1693-47c7-9a11-cfa2249d3635-kube-api-access-crq85" pod "e05c1aac-1693-47c7-9a11-cfa2249d3635" (UID: "e05c1aac-1693-47c7-9a11-cfa2249d3635"). InnerVolumeSpecName "kube-api-access-crq85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:52.240158 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.240133    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"
Oct 30 19:49:52.323596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.323552    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ab659c59-028b-48cb-9510-bdd55e3907fe-kube-api-access-kxtmk\" (UniqueName: \"kubernetes.io/projected/ab659c59-028b-48cb-9510-bdd55e3907fe-kube-api-access-kxtmk\") pod \"ab659c59-028b-48cb-9510-bdd55e3907fe\" (UID: \"ab659c59-028b-48cb-9510-bdd55e3907fe\") "
Oct 30 19:49:52.323768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.323646    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ab659c59-028b-48cb-9510-bdd55e3907fe-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab659c59-028b-48cb-9510-bdd55e3907fe-plugins-config-volume\") pod \"ab659c59-028b-48cb-9510-bdd55e3907fe\" (UID: \"ab659c59-028b-48cb-9510-bdd55e3907fe\") "
Oct 30 19:49:52.323920 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.323909    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crq85\" (UniqueName: \"kubernetes.io/projected/e05c1aac-1693-47c7-9a11-cfa2249d3635-kube-api-access-crq85\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:52.324055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.324012    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab659c59-028b-48cb-9510-bdd55e3907fe-plugins-config-volume" pod "ab659c59-028b-48cb-9510-bdd55e3907fe" (UID: "ab659c59-028b-48cb-9510-bdd55e3907fe"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:49:52.325663 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.325644    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab659c59-028b-48cb-9510-bdd55e3907fe-kube-api-access-kxtmk" pod "ab659c59-028b-48cb-9510-bdd55e3907fe" (UID: "ab659c59-028b-48cb-9510-bdd55e3907fe"). InnerVolumeSpecName "kube-api-access-kxtmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:49:52.425743 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.425633    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxtmk\" (UniqueName: \"kubernetes.io/projected/ab659c59-028b-48cb-9510-bdd55e3907fe-kube-api-access-kxtmk\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:52.425743 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.425697    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab659c59-028b-48cb-9510-bdd55e3907fe-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:49:52.568504 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.568462    5704 generic.go:356] "Generic (PLEG): container finished" podID="ab659c59-028b-48cb-9510-bdd55e3907fe" containerID="448bc8e22f40179e377cdb66a1ef7662c55b292b48d90162d1a94b458da01c25" exitCode=0
Oct 30 19:49:52.568703 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.568519    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"
Oct 30 19:49:52.568703 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.568538    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv" event={"ID":"ab659c59-028b-48cb-9510-bdd55e3907fe","Type":"ContainerDied","Data":"448bc8e22f40179e377cdb66a1ef7662c55b292b48d90162d1a94b458da01c25"}
Oct 30 19:49:52.568703 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.568602    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv" event={"ID":"ab659c59-028b-48cb-9510-bdd55e3907fe","Type":"ContainerDied","Data":"ac33ad6a31304edcd12d936adc5e5c1ef390c8e8994becc29cb13ade43bdfa4a"}
Oct 30 19:49:52.568703 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.568632    5704 scope.go:122] "RemoveContainer" containerID="448bc8e22f40179e377cdb66a1ef7662c55b292b48d90162d1a94b458da01c25"
Oct 30 19:49:52.570153 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.570134    5704 generic.go:356] "Generic (PLEG): container finished" podID="e05c1aac-1693-47c7-9a11-cfa2249d3635" containerID="49de41a7d751293e3d56a495d9e92207f58bacf5f3722cee67432f27a95874db" exitCode=0
Oct 30 19:49:52.570241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.570179    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw" event={"ID":"e05c1aac-1693-47c7-9a11-cfa2249d3635","Type":"ContainerDied","Data":"49de41a7d751293e3d56a495d9e92207f58bacf5f3722cee67432f27a95874db"}
Oct 30 19:49:52.570241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.570219    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw" event={"ID":"e05c1aac-1693-47c7-9a11-cfa2249d3635","Type":"ContainerDied","Data":"410cb40797d9ef42c0a2730de487eefaea161698a4d3ca8d2931d48510aef59b"}
Oct 30 19:49:52.570241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.570182    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"
Oct 30 19:49:52.580368 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.580350    5704 scope.go:122] "RemoveContainer" containerID="448bc8e22f40179e377cdb66a1ef7662c55b292b48d90162d1a94b458da01c25"
Oct 30 19:49:52.580795 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:49:52.580771    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"448bc8e22f40179e377cdb66a1ef7662c55b292b48d90162d1a94b458da01c25\": container with ID starting with 448bc8e22f40179e377cdb66a1ef7662c55b292b48d90162d1a94b458da01c25 not found: ID does not exist" containerID="448bc8e22f40179e377cdb66a1ef7662c55b292b48d90162d1a94b458da01c25"
Oct 30 19:49:52.580911 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.580806    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"448bc8e22f40179e377cdb66a1ef7662c55b292b48d90162d1a94b458da01c25"} err="failed to get container status \"448bc8e22f40179e377cdb66a1ef7662c55b292b48d90162d1a94b458da01c25\": rpc error: code = NotFound desc = could not find container \"448bc8e22f40179e377cdb66a1ef7662c55b292b48d90162d1a94b458da01c25\": container with ID starting with 448bc8e22f40179e377cdb66a1ef7662c55b292b48d90162d1a94b458da01c25 not found: ID does not exist"
Oct 30 19:49:52.580911 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.580824    5704 scope.go:122] "RemoveContainer" containerID="49de41a7d751293e3d56a495d9e92207f58bacf5f3722cee67432f27a95874db"
Oct 30 19:49:52.590201 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.590179    5704 scope.go:122] "RemoveContainer" containerID="49de41a7d751293e3d56a495d9e92207f58bacf5f3722cee67432f27a95874db"
Oct 30 19:49:52.590477 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:49:52.590455    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49de41a7d751293e3d56a495d9e92207f58bacf5f3722cee67432f27a95874db\": container with ID starting with 49de41a7d751293e3d56a495d9e92207f58bacf5f3722cee67432f27a95874db not found: ID does not exist" containerID="49de41a7d751293e3d56a495d9e92207f58bacf5f3722cee67432f27a95874db"
Oct 30 19:49:52.590560 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.590484    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49de41a7d751293e3d56a495d9e92207f58bacf5f3722cee67432f27a95874db"} err="failed to get container status \"49de41a7d751293e3d56a495d9e92207f58bacf5f3722cee67432f27a95874db\": rpc error: code = NotFound desc = could not find container \"49de41a7d751293e3d56a495d9e92207f58bacf5f3722cee67432f27a95874db\": container with ID starting with 49de41a7d751293e3d56a495d9e92207f58bacf5f3722cee67432f27a95874db not found: ID does not exist"
Oct 30 19:49:52.717111 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.716787    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:52.728908 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.728868    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"]
Oct 30 19:49:52.731372 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.731345    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-f858d4-7d67cf6475-skqbv"]
Oct 30 19:49:52.835463 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.835431    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"]
Oct 30 19:49:52.851923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:52.851890    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-f858d4-66bd9c46cd-8zxqw"]
Oct 30 19:49:53.179073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:53.179048    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ab659c59-028b-48cb-9510-bdd55e3907fe" path="/var/lib/kubelet/pods/ab659c59-028b-48cb-9510-bdd55e3907fe/volumes"
Oct 30 19:49:53.179454 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:53.179441    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e05c1aac-1693-47c7-9a11-cfa2249d3635" path="/var/lib/kubelet/pods/e05c1aac-1693-47c7-9a11-cfa2249d3635/volumes"
Oct 30 19:49:57.280472 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:57.280432    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:49:59.636165 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:49:59.636125    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:00.598270 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:00.598228    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:00.793327 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:00.793293    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:00.989561 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:00.989473    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:02.180854 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:02.180794    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:03.384738 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:50:03.384701    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:50:07.171274 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:07.171219    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:11.168824 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:11.168777    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:12.713193 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:12.713150    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:12.908055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:12.904488    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:13.087001 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:13.086958    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:13.604126 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:13.604087    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:13.782312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:13.782275    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:13.962211 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:13.962123    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:15.529181 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:15.529152    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:16.974129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:16.974098    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"]
Oct 30 19:50:16.974548 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:16.974301    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl" podUID="0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b" containerName="maas-api" containerID="cri-o://d50836f917af1fb4b98c18f3ec55318939ed33e58fb72f6b04ed9a43d9532be1" gracePeriod=30
Oct 30 19:50:17.113371 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.113341    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:17.274004 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.273982    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:50:17.337442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.337412    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-openshift-service-ca\") pod \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") "
Oct 30 19:50:17.337599 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.337497    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-kube-api-access-spc2p\" (UniqueName: \"kubernetes.io/projected/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-kube-api-access-spc2p\") pod \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") "
Oct 30 19:50:17.337599 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.337560    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-metrics-certs\") pod \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") "
Oct 30 19:50:17.337757 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.337733    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-maas-api-tls\") pod \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\" (UID: \"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b\") "
Oct 30 19:50:17.337757 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.337734    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-openshift-service-ca" pod "0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b" (UID: "0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:50:17.338154 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.338134    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:17.339615 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.339593    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-metrics-certs" pod "0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b" (UID: "0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:50:17.339715 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.339638    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-kube-api-access-spc2p" pod "0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b" (UID: "0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b"). InnerVolumeSpecName "kube-api-access-spc2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:50:17.339715 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.339649    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-maas-api-tls" pod "0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b" (UID: "0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:50:17.439670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.439632    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:17.439670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.439663    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spc2p\" (UniqueName: \"kubernetes.io/projected/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-kube-api-access-spc2p\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:17.439670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.439678    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:17.749524 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.749492    5704 generic.go:356] "Generic (PLEG): container finished" podID="0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b" containerID="d50836f917af1fb4b98c18f3ec55318939ed33e58fb72f6b04ed9a43d9532be1" exitCode=0
Oct 30 19:50:17.749692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.749530    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"
Oct 30 19:50:17.749692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.749537    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl" event={"ID":"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b","Type":"ContainerDied","Data":"d50836f917af1fb4b98c18f3ec55318939ed33e58fb72f6b04ed9a43d9532be1"}
Oct 30 19:50:17.749692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.749582    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl" event={"ID":"0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b","Type":"ContainerDied","Data":"4b471e964d5ef7320aaa6b4e74ffb1488f5640fe9a837de992497bed795a87b6"}
Oct 30 19:50:17.749692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.749625    5704 scope.go:122] "RemoveContainer" containerID="d50836f917af1fb4b98c18f3ec55318939ed33e58fb72f6b04ed9a43d9532be1"
Oct 30 19:50:17.766180 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.766155    5704 scope.go:122] "RemoveContainer" containerID="d50836f917af1fb4b98c18f3ec55318939ed33e58fb72f6b04ed9a43d9532be1"
Oct 30 19:50:17.766565 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:50:17.766533    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d50836f917af1fb4b98c18f3ec55318939ed33e58fb72f6b04ed9a43d9532be1\": container with ID starting with d50836f917af1fb4b98c18f3ec55318939ed33e58fb72f6b04ed9a43d9532be1 not found: ID does not exist" containerID="d50836f917af1fb4b98c18f3ec55318939ed33e58fb72f6b04ed9a43d9532be1"
Oct 30 19:50:17.766682 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.766577    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d50836f917af1fb4b98c18f3ec55318939ed33e58fb72f6b04ed9a43d9532be1"} err="failed to get container status \"d50836f917af1fb4b98c18f3ec55318939ed33e58fb72f6b04ed9a43d9532be1\": rpc error: code = NotFound desc = could not find container \"d50836f917af1fb4b98c18f3ec55318939ed33e58fb72f6b04ed9a43d9532be1\": container with ID starting with d50836f917af1fb4b98c18f3ec55318939ed33e58fb72f6b04ed9a43d9532be1 not found: ID does not exist"
Oct 30 19:50:17.910829 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.910645    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"]
Oct 30 19:50:17.955288 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:17.955251    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-praxis-b-a2f378-59d8f4f547-vlmcl"]
Oct 30 19:50:18.647830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:18.647800    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:18.876280 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:18.876241    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:19.180395 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:19.180364    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b" path="/var/lib/kubelet/pods/0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b/volumes"
Oct 30 19:50:19.501911 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:19.501878    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:20.552541 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:20.552509    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:20.754340 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:20.754294    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:20.991941 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:20.991846    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:21.562371 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:21.562331    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:22.167815 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:22.167777    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:22.405341 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:22.405306    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:23.727164 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:23.726683    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:24.484016 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:24.483983    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:24.692592 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:24.692559    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:24.873712 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:24.873677    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:27.141936 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:27.137908    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:27.329125 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:27.329086    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:27.522352 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:27.522321    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"]
Oct 30 19:50:27.522544 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:27.522524    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6" podUID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerName="istio-proxy" containerID="cri-o://bd1b56426fbe99b3d7ecf399ea845cfb10dacd1c1e6a633982fe1126efbc89a1" gracePeriod=30
Oct 30 19:50:27.574512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:27.574481    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:28.465958 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:28.465922    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:28.655979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:28.655939    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:29.089292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:29.089258    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:29.370506 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:29.370425    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:32.790823 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.790767    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:50:32.849872 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.849842    5704 generic.go:356] "Generic (PLEG): container finished" podID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerID="bd1b56426fbe99b3d7ecf399ea845cfb10dacd1c1e6a633982fe1126efbc89a1" exitCode=0
Oct 30 19:50:32.850054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.849894    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"
Oct 30 19:50:32.850054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.849922    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6" event={"ID":"bcb4ec43-aa95-4bad-b9fb-71fb87308901","Type":"ContainerDied","Data":"bd1b56426fbe99b3d7ecf399ea845cfb10dacd1c1e6a633982fe1126efbc89a1"}
Oct 30 19:50:32.850054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.849974    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6" event={"ID":"bcb4ec43-aa95-4bad-b9fb-71fb87308901","Type":"ContainerDied","Data":"c783a189681f7ec7c2e53d3f326a9450055c74659d23af89ca34ee229b970fc9"}
Oct 30 19:50:32.850054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.849998    5704 scope.go:122] "RemoveContainer" containerID="bd1b56426fbe99b3d7ecf399ea845cfb10dacd1c1e6a633982fe1126efbc89a1"
Oct 30 19:50:32.859862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.859846    5704 scope.go:122] "RemoveContainer" containerID="f19556b3af32002a6f6999037c4200fb21c5d1c629791e2c5cad0605cfedb4a0"
Oct 30 19:50:32.868503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.868487    5704 scope.go:122] "RemoveContainer" containerID="bd1b56426fbe99b3d7ecf399ea845cfb10dacd1c1e6a633982fe1126efbc89a1"
Oct 30 19:50:32.868760 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:50:32.868745    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd1b56426fbe99b3d7ecf399ea845cfb10dacd1c1e6a633982fe1126efbc89a1\": container with ID starting with bd1b56426fbe99b3d7ecf399ea845cfb10dacd1c1e6a633982fe1126efbc89a1 not found: ID does not exist" containerID="bd1b56426fbe99b3d7ecf399ea845cfb10dacd1c1e6a633982fe1126efbc89a1"
Oct 30 19:50:32.868808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.868767    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd1b56426fbe99b3d7ecf399ea845cfb10dacd1c1e6a633982fe1126efbc89a1"} err="failed to get container status \"bd1b56426fbe99b3d7ecf399ea845cfb10dacd1c1e6a633982fe1126efbc89a1\": rpc error: code = NotFound desc = could not find container \"bd1b56426fbe99b3d7ecf399ea845cfb10dacd1c1e6a633982fe1126efbc89a1\": container with ID starting with bd1b56426fbe99b3d7ecf399ea845cfb10dacd1c1e6a633982fe1126efbc89a1 not found: ID does not exist"
Oct 30 19:50:32.868808 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.868784    5704 scope.go:122] "RemoveContainer" containerID="f19556b3af32002a6f6999037c4200fb21c5d1c629791e2c5cad0605cfedb4a0"
Oct 30 19:50:32.869010 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:50:32.868991    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f19556b3af32002a6f6999037c4200fb21c5d1c629791e2c5cad0605cfedb4a0\": container with ID starting with f19556b3af32002a6f6999037c4200fb21c5d1c629791e2c5cad0605cfedb4a0 not found: ID does not exist" containerID="f19556b3af32002a6f6999037c4200fb21c5d1c629791e2c5cad0605cfedb4a0"
Oct 30 19:50:32.869133 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.869009    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f19556b3af32002a6f6999037c4200fb21c5d1c629791e2c5cad0605cfedb4a0"} err="failed to get container status \"f19556b3af32002a6f6999037c4200fb21c5d1c629791e2c5cad0605cfedb4a0\": rpc error: code = NotFound desc = could not find container \"f19556b3af32002a6f6999037c4200fb21c5d1c629791e2c5cad0605cfedb4a0\": container with ID starting with f19556b3af32002a6f6999037c4200fb21c5d1c629791e2c5cad0605cfedb4a0 not found: ID does not exist"
Oct 30 19:50:32.942662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.942629    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-kube-api-access-kdzk6\" (UniqueName: \"kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-kube-api-access-kdzk6\") pod \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") "
Oct 30 19:50:32.942830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.942801    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istiod-ca-cert\") pod \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") "
Oct 30 19:50:32.942897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.942882    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-credential-socket\") pod \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") "
Oct 30 19:50:32.942936 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.942917    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-data\") pod \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") "
Oct 30 19:50:32.943060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943015    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-socket\") pod \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") "
Oct 30 19:50:32.943142 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943129    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-podinfo\") pod \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") "
Oct 30 19:50:32.943198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943177    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-certs\") pod \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") "
Oct 30 19:50:32.943263 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943208    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-envoy\") pod \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") "
Oct 30 19:50:32.943263 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943234    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-token\" (UniqueName: \"kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-token\") pod \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\" (UID: \"bcb4ec43-aa95-4bad-b9fb-71fb87308901\") "
Oct 30 19:50:32.943379 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943230    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-credential-socket" pod "bcb4ec43-aa95-4bad-b9fb-71fb87308901" (UID: "bcb4ec43-aa95-4bad-b9fb-71fb87308901"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:32.943477 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943453    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-certs" pod "bcb4ec43-aa95-4bad-b9fb-71fb87308901" (UID: "bcb4ec43-aa95-4bad-b9fb-71fb87308901"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:32.943536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943245    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-data" pod "bcb4ec43-aa95-4bad-b9fb-71fb87308901" (UID: "bcb4ec43-aa95-4bad-b9fb-71fb87308901"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:32.943536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943283    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istiod-ca-cert" pod "bcb4ec43-aa95-4bad-b9fb-71fb87308901" (UID: "bcb4ec43-aa95-4bad-b9fb-71fb87308901"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:50:32.943651 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943566    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-socket" pod "bcb4ec43-aa95-4bad-b9fb-71fb87308901" (UID: "bcb4ec43-aa95-4bad-b9fb-71fb87308901"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:32.943954 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943934    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:32.944011 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943960    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:32.944011 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943977    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:32.944011 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.943991    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:32.944011 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.944007    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:32.944994 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.944973    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-kube-api-access-kdzk6" pod "bcb4ec43-aa95-4bad-b9fb-71fb87308901" (UID: "bcb4ec43-aa95-4bad-b9fb-71fb87308901"). InnerVolumeSpecName "kube-api-access-kdzk6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:50:32.945224 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.945204    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-envoy" pod "bcb4ec43-aa95-4bad-b9fb-71fb87308901" (UID: "bcb4ec43-aa95-4bad-b9fb-71fb87308901"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:50:32.945580 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.945560    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-podinfo" pod "bcb4ec43-aa95-4bad-b9fb-71fb87308901" (UID: "bcb4ec43-aa95-4bad-b9fb-71fb87308901"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:50:32.945635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:32.945599    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-token" pod "bcb4ec43-aa95-4bad-b9fb-71fb87308901" (UID: "bcb4ec43-aa95-4bad-b9fb-71fb87308901"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:50:33.045631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:33.045562    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:33.045631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:33.045587    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:33.045631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:33.045596    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:33.045631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:33.045607    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kdzk6\" (UniqueName: \"kubernetes.io/projected/bcb4ec43-aa95-4bad-b9fb-71fb87308901-kube-api-access-kdzk6\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:50:33.265453 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:33.265422    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"]
Oct 30 19:50:33.269612 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:33.269587    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-praxis-b-a2f378-openshift-default-5b755bc685-r85c6"]
Oct 30 19:50:33.388766 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:33.388690    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:33.403541 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:50:33.403508    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:50:33.745088 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:33.744981    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:33.922885 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:33.922837    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:34.095274 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:34.095240    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:35.181013 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:35.180979    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" path="/var/lib/kubelet/pods/bcb4ec43-aa95-4bad-b9fb-71fb87308901/volumes"
Oct 30 19:50:36.527065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:36.527010    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:37.854597 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:37.854562    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:38.202870 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.202792    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:38.491840 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.491757    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:38.708385 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.708354    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:38.893783 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.893717    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"]
Oct 30 19:50:38.895159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895131    5704 cpu_manager.go:414] "containerMap: removing container" podUID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" containerName="cleanup"
Oct 30 19:50:38.895159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895159    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" containerName="cleanup"
Oct 30 19:50:38.895357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895194    5704 cpu_manager.go:414] "containerMap: removing container" podUID="e05c1aac-1693-47c7-9a11-cfa2249d3635" containerName="payload-processing"
Oct 30 19:50:38.895357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895202    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e05c1aac-1693-47c7-9a11-cfa2249d3635" containerName="payload-processing"
Oct 30 19:50:38.895357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895243    5704 cpu_manager.go:414] "containerMap: removing container" podUID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" containerName="cleanup"
Oct 30 19:50:38.895357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895250    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" containerName="cleanup"
Oct 30 19:50:38.895357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895264    5704 cpu_manager.go:414] "containerMap: removing container" podUID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerName="istio-proxy"
Oct 30 19:50:38.895357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895271    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerName="istio-proxy"
Oct 30 19:50:38.895357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895287    5704 cpu_manager.go:414] "containerMap: removing container" podUID="ab659c59-028b-48cb-9510-bdd55e3907fe" containerName="payload-pre-processing"
Oct 30 19:50:38.895357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895294    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ab659c59-028b-48cb-9510-bdd55e3907fe" containerName="payload-pre-processing"
Oct 30 19:50:38.895357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895357    5704 cpu_manager.go:414] "containerMap: removing container" podUID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" containerName="istio-proxy"
Oct 30 19:50:38.895787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895364    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" containerName="istio-proxy"
Oct 30 19:50:38.895787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895393    5704 cpu_manager.go:414] "containerMap: removing container" podUID="4ea5bfd4-5c12-462d-beb8-2944091571f5" containerName="payload-processing"
Oct 30 19:50:38.895787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895400    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ea5bfd4-5c12-462d-beb8-2944091571f5" containerName="payload-processing"
Oct 30 19:50:38.895787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895436    5704 cpu_manager.go:414] "containerMap: removing container" podUID="0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b" containerName="maas-api"
Oct 30 19:50:38.895787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895442    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b" containerName="maas-api"
Oct 30 19:50:38.895787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895698    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" containerName="cleanup"
Oct 30 19:50:38.895787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895720    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="e05c1aac-1693-47c7-9a11-cfa2249d3635" containerName="payload-processing"
Oct 30 19:50:38.895787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895737    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" containerName="cleanup"
Oct 30 19:50:38.895787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895778    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerName="istio-proxy"
Oct 30 19:50:38.895787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895787    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="ab659c59-028b-48cb-9510-bdd55e3907fe" containerName="payload-pre-processing"
Oct 30 19:50:38.896259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895812    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="c1036dd5-2b14-45fb-a368-b3ef1fccc7c6" containerName="istio-proxy"
Oct 30 19:50:38.896259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895822    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" containerName="cleanup"
Oct 30 19:50:38.896259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895835    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ea5bfd4-5c12-462d-beb8-2944091571f5" containerName="payload-processing"
Oct 30 19:50:38.896259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895854    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerName="istio-proxy"
Oct 30 19:50:38.896259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.895871    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="0af5e3bf-9e24-4a4a-a591-f2dbda8ccd5b" containerName="maas-api"
Oct 30 19:50:38.900601 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.900570    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:38.903836 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.903812    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"
Oct 30 19:50:38.961106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:38.961078    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-b330be\""
Oct 30 19:50:39.003237 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.003202    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-b330be-dockercfg-2svsx\""
Oct 30 19:50:39.006560 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.006535    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"]
Oct 30 19:50:39.047070 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.036810    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxzlv\" (UniqueName: \"kubernetes.io/projected/185e2fc0-40ef-4f92-87fa-d2257741f604-kube-api-access-mxzlv\") pod \"payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6\" (UID: \"185e2fc0-40ef-4f92-87fa-d2257741f604\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"
Oct 30 19:50:39.047296 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.045708    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"]
Oct 30 19:50:39.047566 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.047549    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/185e2fc0-40ef-4f92-87fa-d2257741f604-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6\" (UID: \"185e2fc0-40ef-4f92-87fa-d2257741f604\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"
Oct 30 19:50:39.048351 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.048330    5704 cpu_manager.go:414] "containerMap: removing container" podUID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" containerName="cleanup"
Oct 30 19:50:39.048451 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.048353    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c2e2ea6-1fb2-4bfd-80a5-425dcd4b7622" containerName="cleanup"
Oct 30 19:50:39.048451 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.048375    5704 cpu_manager.go:414] "containerMap: removing container" podUID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerName="istio-proxy"
Oct 30 19:50:39.048451 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.048380    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bcb4ec43-aa95-4bad-b9fb-71fb87308901" containerName="istio-proxy"
Oct 30 19:50:39.052550 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.052530    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"
Oct 30 19:50:39.100652 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.100623    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"]
Oct 30 19:50:39.111837 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.111804    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:39.148517 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.148447    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxzlv\" (UniqueName: \"kubernetes.io/projected/185e2fc0-40ef-4f92-87fa-d2257741f604-kube-api-access-mxzlv\") pod \"payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6\" (UID: \"185e2fc0-40ef-4f92-87fa-d2257741f604\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"
Oct 30 19:50:39.148646 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.148518    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-plugins-config-volume\") pod \"payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt\" (UID: \"f26b13a2-df3c-4e26-a7af-0bd40ddd3190\") " pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"
Oct 30 19:50:39.148723 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.148705    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/185e2fc0-40ef-4f92-87fa-d2257741f604-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6\" (UID: \"185e2fc0-40ef-4f92-87fa-d2257741f604\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"
Oct 30 19:50:39.148830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.148806    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5txxs\" (UniqueName: \"kubernetes.io/projected/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-kube-api-access-5txxs\") pod \"payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt\" (UID: \"f26b13a2-df3c-4e26-a7af-0bd40ddd3190\") " pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"
Oct 30 19:50:39.149395 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.149377    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/185e2fc0-40ef-4f92-87fa-d2257741f604-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6\" (UID: \"185e2fc0-40ef-4f92-87fa-d2257741f604\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"
Oct 30 19:50:39.219997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.219967    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxzlv\" (UniqueName: \"kubernetes.io/projected/185e2fc0-40ef-4f92-87fa-d2257741f604-kube-api-access-mxzlv\") pod \"payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6\" (UID: \"185e2fc0-40ef-4f92-87fa-d2257741f604\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"
Oct 30 19:50:39.220763 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.220743    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"
Oct 30 19:50:39.249682 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.249647    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5txxs\" (UniqueName: \"kubernetes.io/projected/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-kube-api-access-5txxs\") pod \"payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt\" (UID: \"f26b13a2-df3c-4e26-a7af-0bd40ddd3190\") " pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"
Oct 30 19:50:39.249810 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.249728    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-plugins-config-volume\") pod \"payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt\" (UID: \"f26b13a2-df3c-4e26-a7af-0bd40ddd3190\") " pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"
Oct 30 19:50:39.250446 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.250422    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-plugins-config-volume\") pod \"payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt\" (UID: \"f26b13a2-df3c-4e26-a7af-0bd40ddd3190\") " pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"
Oct 30 19:50:39.311878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.311854    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5txxs\" (UniqueName: \"kubernetes.io/projected/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-kube-api-access-5txxs\") pod \"payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt\" (UID: \"f26b13a2-df3c-4e26-a7af-0bd40ddd3190\") " pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"
Oct 30 19:50:39.373901 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.373873    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"
Oct 30 19:50:39.597504 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.597480    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"]
Oct 30 19:50:39.600195 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:50:39.600168    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod185e2fc0_40ef_4f92_87fa_d2257741f604.slice/crio-b291d979dc453e6df1b2b7e3593a5dddadd72a87b630b41825144ad7850c403f WatchSource:0}: Error finding container b291d979dc453e6df1b2b7e3593a5dddadd72a87b630b41825144ad7850c403f: Status 404 returned error can't find the container with id b291d979dc453e6df1b2b7e3593a5dddadd72a87b630b41825144ad7850c403f
Oct 30 19:50:39.745549 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.745520    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"]
Oct 30 19:50:39.747858 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:50:39.747829    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf26b13a2_df3c_4e26_a7af_0bd40ddd3190.slice/crio-530c7cf828afe0babf04ac8502ff263f204beab9a102773cd94bf93fdecf68d3 WatchSource:0}: Error finding container 530c7cf828afe0babf04ac8502ff263f204beab9a102773cd94bf93fdecf68d3: Status 404 returned error can't find the container with id 530c7cf828afe0babf04ac8502ff263f204beab9a102773cd94bf93fdecf68d3
Oct 30 19:50:39.896914 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.896879    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt" event={"ID":"f26b13a2-df3c-4e26-a7af-0bd40ddd3190","Type":"ContainerStarted","Data":"91b12a9707c0dfe322d78dacd58555470d3d3924f4afd8e2be46cc03ee00f6ce"}
Oct 30 19:50:39.897332 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.896934    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt" event={"ID":"f26b13a2-df3c-4e26-a7af-0bd40ddd3190","Type":"ContainerStarted","Data":"530c7cf828afe0babf04ac8502ff263f204beab9a102773cd94bf93fdecf68d3"}
Oct 30 19:50:39.897332 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.897063    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"
Oct 30 19:50:39.898642 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.898620    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6" event={"ID":"185e2fc0-40ef-4f92-87fa-d2257741f604","Type":"ContainerStarted","Data":"e41b14b826fcba55c156d0f5318002c8b5b5e032375ba8d5d4355c5c3b6b5d29"}
Oct 30 19:50:39.898784 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.898653    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6" event={"ID":"185e2fc0-40ef-4f92-87fa-d2257741f604","Type":"ContainerStarted","Data":"b291d979dc453e6df1b2b7e3593a5dddadd72a87b630b41825144ad7850c403f"}
Oct 30 19:50:39.898784 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:39.898757    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"
Oct 30 19:50:40.007398 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:40.007347    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt" podStartSLOduration=2.007330139 podStartE2EDuration="2.007330139s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:50:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:50:40.005634958 +0000 UTC m=+3812.951802862" watchObservedRunningTime="2026-10-30 19:50:40.007330139 +0000 UTC m=+3812.953498073"
Oct 30 19:50:40.118008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:40.117956    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6" podStartSLOduration=2.117941597 podStartE2EDuration="2.117941597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:50:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:50:40.115303413 +0000 UTC m=+3813.061471318" watchObservedRunningTime="2026-10-30 19:50:40.117941597 +0000 UTC m=+3813.064109499"
Oct 30 19:50:42.769757 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:42.769724    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:42.956803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:42.956763    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:43.139060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:43.138965    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:45.892875 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:45.892839    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:46.067487 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:46.067453    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:46.441749 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:46.441049    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"]
Oct 30 19:50:46.449305 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:46.449274    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"
Oct 30 19:50:46.492876 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:46.492847    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"]
Oct 30 19:50:46.498022 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:46.497999    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-4a36b7\""
Oct 30 19:50:46.498228 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:46.498207    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-4a36b7-dockercfg-cq9kc\""
Oct 30 19:50:46.545921 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:46.545888    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gb7m\" (UniqueName: \"kubernetes.io/projected/f9b0ac2e-2d36-454f-8666-25acd2696603-kube-api-access-9gb7m\") pod \"payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf\" (UID: \"f9b0ac2e-2d36-454f-8666-25acd2696603\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"
Oct 30 19:50:46.546121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:46.546019    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b0ac2e-2d36-454f-8666-25acd2696603-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf\" (UID: \"f9b0ac2e-2d36-454f-8666-25acd2696603\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"
Oct 30 19:50:46.647350 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:46.647319    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gb7m\" (UniqueName: \"kubernetes.io/projected/f9b0ac2e-2d36-454f-8666-25acd2696603-kube-api-access-9gb7m\") pod \"payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf\" (UID: \"f9b0ac2e-2d36-454f-8666-25acd2696603\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"
Oct 30 19:50:46.647478 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:46.647411    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b0ac2e-2d36-454f-8666-25acd2696603-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf\" (UID: \"f9b0ac2e-2d36-454f-8666-25acd2696603\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"
Oct 30 19:50:46.648160 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:46.648140    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b0ac2e-2d36-454f-8666-25acd2696603-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf\" (UID: \"f9b0ac2e-2d36-454f-8666-25acd2696603\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"
Oct 30 19:50:46.705459 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:46.705403    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gb7m\" (UniqueName: \"kubernetes.io/projected/f9b0ac2e-2d36-454f-8666-25acd2696603-kube-api-access-9gb7m\") pod \"payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf\" (UID: \"f9b0ac2e-2d36-454f-8666-25acd2696603\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"
Oct 30 19:50:46.767020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:46.766995    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"
Oct 30 19:50:47.146503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:47.146482    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"]
Oct 30 19:50:47.148945 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:50:47.148921    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b0ac2e_2d36_454f_8666_25acd2696603.slice/crio-5f0e5c3c75f061610b59862cb72055aef0c4646b237242957426b035bc5b326a WatchSource:0}: Error finding container 5f0e5c3c75f061610b59862cb72055aef0c4646b237242957426b035bc5b326a: Status 404 returned error can't find the container with id 5f0e5c3c75f061610b59862cb72055aef0c4646b237242957426b035bc5b326a
Oct 30 19:50:47.823825 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:47.823789    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:47.950252 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:47.950216    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf" event={"ID":"f9b0ac2e-2d36-454f-8666-25acd2696603","Type":"ContainerStarted","Data":"14e51e7912b90137b0167c88eff4ee26433a0ff350bcfb8a69704ccac3301948"}
Oct 30 19:50:47.950442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:47.950273    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf" event={"ID":"f9b0ac2e-2d36-454f-8666-25acd2696603","Type":"ContainerStarted","Data":"5f0e5c3c75f061610b59862cb72055aef0c4646b237242957426b035bc5b326a"}
Oct 30 19:50:47.950442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:47.950323    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"
Oct 30 19:50:48.011778 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:48.011741    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:48.062028 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:48.061976    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf" podStartSLOduration=2.061958992 podStartE2EDuration="2.061958992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:50:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:50:48.060940392 +0000 UTC m=+3821.007108296" watchObservedRunningTime="2026-10-30 19:50:48.061958992 +0000 UTC m=+3821.008126894"
Oct 30 19:50:48.200313 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:48.199723    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:50.908638 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:50.908612    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"
Oct 30 19:50:50.909162 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:50.908772    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"
Oct 30 19:50:52.709698 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:52.709653    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:56.690542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:56.690512    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:50:57.417080 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:57.417026    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:50:58.960016 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:50:58.959985    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"
Oct 30 19:51:00.839333 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:00.839298    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:01.988709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:01.988671    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:03.383203 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:51:03.383170    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:51:03.605262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:03.605225    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:06.665476 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:06.665436    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:11.263241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:11.263205    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:15.816328 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:15.816284    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:16.007442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:16.007401    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:18.880097 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:18.880061    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:20.496102 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:20.496065    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:20.674512 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:20.674477    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:21.821616 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:21.821572    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:22.118278 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:22.118179    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:24.890265 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:24.890229    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:25.090427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:25.090391    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:25.292800 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:25.292762    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:26.055867 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:26.055833    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:26.307138 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:26.307016    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:27.581947 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:27.581913    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:27.762153 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:27.762114    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:28.098719 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:28.098684    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:28.289198 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:28.289165    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:29.174224 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.174189    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:29.463667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.463580    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:29.545314 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.545281    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"]
Oct 30 19:51:29.549814 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.549794    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"
Oct 30 19:51:29.596015 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.595985    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5rw5\" (UniqueName: \"kubernetes.io/projected/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-kube-api-access-t5rw5\") pod \"payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f\" (UID: \"6123ea10-1fe2-482b-9f1d-e6523f98cd7b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"
Oct 30 19:51:29.596177 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.596107    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f\" (UID: \"6123ea10-1fe2-482b-9f1d-e6523f98cd7b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"
Oct 30 19:51:29.598577 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.598552    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-a-905752\""
Oct 30 19:51:29.645334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.645299    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-a-905752-dockercfg-2942d\""
Oct 30 19:51:29.674459 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.658606    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"]
Oct 30 19:51:29.698016 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.697983    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5rw5\" (UniqueName: \"kubernetes.io/projected/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-kube-api-access-t5rw5\") pod \"payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f\" (UID: \"6123ea10-1fe2-482b-9f1d-e6523f98cd7b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"
Oct 30 19:51:29.698267 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.698087    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f\" (UID: \"6123ea10-1fe2-482b-9f1d-e6523f98cd7b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"
Oct 30 19:51:29.698853 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.698829    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f\" (UID: \"6123ea10-1fe2-482b-9f1d-e6523f98cd7b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"
Oct 30 19:51:29.705555 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.705527    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"]
Oct 30 19:51:29.709981 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.709959    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"
Oct 30 19:51:29.756046 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.756002    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5rw5\" (UniqueName: \"kubernetes.io/projected/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-kube-api-access-t5rw5\") pod \"payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f\" (UID: \"6123ea10-1fe2-482b-9f1d-e6523f98cd7b\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"
Oct 30 19:51:29.777321 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.777290    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"]
Oct 30 19:51:29.799651 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.799620    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpqcm\" (UniqueName: \"kubernetes.io/projected/7dd417f3-3db2-401a-abdf-15fff44fc711-kube-api-access-qpqcm\") pod \"payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll\" (UID: \"7dd417f3-3db2-401a-abdf-15fff44fc711\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"
Oct 30 19:51:29.799809 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.799659    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dd417f3-3db2-401a-abdf-15fff44fc711-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll\" (UID: \"7dd417f3-3db2-401a-abdf-15fff44fc711\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"
Oct 30 19:51:29.863911 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.863880    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"
Oct 30 19:51:29.900710 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.900673    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpqcm\" (UniqueName: \"kubernetes.io/projected/7dd417f3-3db2-401a-abdf-15fff44fc711-kube-api-access-qpqcm\") pod \"payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll\" (UID: \"7dd417f3-3db2-401a-abdf-15fff44fc711\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"
Oct 30 19:51:29.900710 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.900714    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dd417f3-3db2-401a-abdf-15fff44fc711-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll\" (UID: \"7dd417f3-3db2-401a-abdf-15fff44fc711\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"
Oct 30 19:51:29.901499 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.901475    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dd417f3-3db2-401a-abdf-15fff44fc711-plugins-config-volume\") pod \"payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll\" (UID: \"7dd417f3-3db2-401a-abdf-15fff44fc711\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"
Oct 30 19:51:29.958542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:29.958504    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpqcm\" (UniqueName: \"kubernetes.io/projected/7dd417f3-3db2-401a-abdf-15fff44fc711-kube-api-access-qpqcm\") pod \"payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll\" (UID: \"7dd417f3-3db2-401a-abdf-15fff44fc711\") " pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"
Oct 30 19:51:30.023368 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:30.023341    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"
Oct 30 19:51:30.237419 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:30.237393    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"]
Oct 30 19:51:30.239557 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:51:30.239533    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6123ea10_1fe2_482b_9f1d_e6523f98cd7b.slice/crio-a456dc7b7d54a9a8021b28211ea404096c7a1eeecb25ef4b1712d8d46d1164f4 WatchSource:0}: Error finding container a456dc7b7d54a9a8021b28211ea404096c7a1eeecb25ef4b1712d8d46d1164f4: Status 404 returned error can't find the container with id a456dc7b7d54a9a8021b28211ea404096c7a1eeecb25ef4b1712d8d46d1164f4
Oct 30 19:51:30.396801 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:30.396776    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"]
Oct 30 19:51:30.400387 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:51:30.400359    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd417f3_3db2_401a_abdf_15fff44fc711.slice/crio-9d6c56b4597777ec2b93eeca45a402f1f01c2457960dc77118e36a52c64ff6d9 WatchSource:0}: Error finding container 9d6c56b4597777ec2b93eeca45a402f1f01c2457960dc77118e36a52c64ff6d9: Status 404 returned error can't find the container with id 9d6c56b4597777ec2b93eeca45a402f1f01c2457960dc77118e36a52c64ff6d9
Oct 30 19:51:30.422341 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:30.422250    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:31.220112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:31.220079    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll" event={"ID":"7dd417f3-3db2-401a-abdf-15fff44fc711","Type":"ContainerStarted","Data":"a3b16677194275b337043f55d1249c6c71395e81dae671d6ad5acb853e155939"}
Oct 30 19:51:31.220313 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:31.220126    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll" event={"ID":"7dd417f3-3db2-401a-abdf-15fff44fc711","Type":"ContainerStarted","Data":"9d6c56b4597777ec2b93eeca45a402f1f01c2457960dc77118e36a52c64ff6d9"}
Oct 30 19:51:31.220313 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:31.220154    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"
Oct 30 19:51:31.221780 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:31.221756    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f" event={"ID":"6123ea10-1fe2-482b-9f1d-e6523f98cd7b","Type":"ContainerStarted","Data":"2845e97cb587fde1a1b464d10fc9f231c2ab177ddc75535219ed4fb3e8f6b08b"}
Oct 30 19:51:31.221901 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:31.221795    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f" event={"ID":"6123ea10-1fe2-482b-9f1d-e6523f98cd7b","Type":"ContainerStarted","Data":"a456dc7b7d54a9a8021b28211ea404096c7a1eeecb25ef4b1712d8d46d1164f4"}
Oct 30 19:51:31.221901 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:31.221849    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"
Oct 30 19:51:31.335470 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:31.335416    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll" podStartSLOduration=2.335399742 podStartE2EDuration="2.335399742s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:51:31.334874233 +0000 UTC m=+3864.281042134" watchObservedRunningTime="2026-10-30 19:51:31.335399742 +0000 UTC m=+3864.281567646"
Oct 30 19:51:31.446159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:31.446106    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f" podStartSLOduration=2.446092537 podStartE2EDuration="2.446092537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:51:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:51:31.444510149 +0000 UTC m=+3864.390678052" watchObservedRunningTime="2026-10-30 19:51:31.446092537 +0000 UTC m=+3864.392260446"
Oct 30 19:51:33.318713 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:33.318680    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:33.382990 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:51:33.382960    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:51:35.505833 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:35.505799    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:40.542127 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:40.542093    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:42.230773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:42.230744    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"
Oct 30 19:51:42.231228 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:42.230871    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"
Oct 30 19:51:45.214337 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:45.214301    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:45.316110 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:45.316073    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:51:49.838475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:49.838389    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:50.027662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:50.027632    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:54.562706 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:54.562664    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:51:59.183334 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:51:59.183154    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:03.384279 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:52:03.384244    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:52:03.472803 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.472771    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"]
Oct 30 19:52:03.476825 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.476805    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:03.525718 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.525693    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-b-4a36b7-serving-cert\""
Oct 30 19:52:03.575947 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.575911    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-maas-api-tls\") pod \"maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:03.576169 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.575957    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-metrics-certs\") pod \"maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:03.576255 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.576111    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa9d942-3670-49ef-b4bc-a3fe995a642f-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:03.576414 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.576392    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mnjsq\" (UniqueName: \"kubernetes.io/projected/faa9d942-3670-49ef-b4bc-a3fe995a642f-kube-api-access-mnjsq\") pod \"maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:03.578122 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.578096    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"]
Oct 30 19:52:03.666494 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.666430    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:03.689526 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.689489    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-maas-api-tls\") pod \"maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:03.689738 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.689536    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-metrics-certs\") pod \"maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:03.689738 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.689618    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa9d942-3670-49ef-b4bc-a3fe995a642f-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:03.689894 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.689765    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mnjsq\" (UniqueName: \"kubernetes.io/projected/faa9d942-3670-49ef-b4bc-a3fe995a642f-kube-api-access-mnjsq\") pod \"maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:03.690303 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:52:03.690281    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-serving-cert: secret "maas-api-e2e-extmod-b-4a36b7-serving-cert" not found
Oct 30 19:52:03.691095 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:52:03.691079    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-maas-api-tls podName:faa9d942-3670-49ef-b4bc-a3fe995a642f nodeName:}" failed. No retries permitted until 2026-10-30 19:52:04.191014413 +0000 UTC m=+3897.137182305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-maas-api-tls") pod "maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l" (UID: "faa9d942-3670-49ef-b4bc-a3fe995a642f") : secret "maas-api-e2e-extmod-b-4a36b7-serving-cert" not found
Oct 30 19:52:03.697195 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.694141    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa9d942-3670-49ef-b4bc-a3fe995a642f-openshift-service-ca\") pod \"maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:03.702694 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.702666    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-metrics-certs\") pod \"maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:03.747680 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.747652    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mnjsq\" (UniqueName: \"kubernetes.io/projected/faa9d942-3670-49ef-b4bc-a3fe995a642f-kube-api-access-mnjsq\") pod \"maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:03.874645 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:03.874608    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:04.048480 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:04.048447    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:04.196441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:04.196411    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-maas-api-tls\") pod \"maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:04.198842 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:04.198818    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-maas-api-tls\") pod \"maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:04.390768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:04.390691    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:04.763079 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:04.763051    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"]
Oct 30 19:52:04.765279 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:52:04.765248    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaa9d942_3670_49ef_b4bc_a3fe995a642f.slice/crio-dae5829ef140dc8d23d843559046fa897129490abc308e6fd06ad86eb4eb5bd3 WatchSource:0}: Error finding container dae5829ef140dc8d23d843559046fa897129490abc308e6fd06ad86eb4eb5bd3: Status 404 returned error can't find the container with id dae5829ef140dc8d23d843559046fa897129490abc308e6fd06ad86eb4eb5bd3
Oct 30 19:52:04.767048 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:04.767018    5704 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 19:52:05.444310 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:05.444274    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l" event={"ID":"faa9d942-3670-49ef-b4bc-a3fe995a642f","Type":"ContainerStarted","Data":"dae5829ef140dc8d23d843559046fa897129490abc308e6fd06ad86eb4eb5bd3"}
Oct 30 19:52:06.453619 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:06.453578    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l" event={"ID":"faa9d942-3670-49ef-b4bc-a3fe995a642f","Type":"ContainerStarted","Data":"ebe4b06d0c3acc28bd11a4c65faf557d7de85f527ed55ac80ec0e4a60df61d8f"}
Oct 30 19:52:06.454070 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:06.453649    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:06.566940 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:06.566883    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l" podStartSLOduration=2.735532918 podStartE2EDuration="3.566865841s" totalImagesPullingTime="831.332923ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:06.565158311 +0000 UTC m=+3899.511326214" watchObservedRunningTime="2026-10-30 19:52:06.566865841 +0000 UTC m=+3899.513033746"
Oct 30 19:52:08.661899 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:08.661852    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:11.579168 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:11.579128    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:12.466913 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:12.466885    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:52:13.291877 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:13.291847    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:13.482944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:13.482909    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:13.930929 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:13.930894    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:14.253276 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:14.253247    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:14.905422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:14.905390    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"]
Oct 30 19:52:14.909732 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:14.909711    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:14.958861 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:14.958840    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-b330be-openshift-default-dockercfg-w965c\""
Oct 30 19:52:15.029725 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.029694    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-podinfo\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.029884 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.029735    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istiod-ca-cert\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.029884 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.029815    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mkl9\" (UniqueName: \"kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-kube-api-access-5mkl9\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.029969 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.029910    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-socket\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.029969 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.029938    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-certs\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.030074 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.029972    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-envoy\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.030074 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.030063    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-data\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.030175 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.030102    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-credential-socket\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.030243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.030226    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-token\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.110815 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.110779    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"]
Oct 30 19:52:15.131630 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.131595    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-socket\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.131807 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.131633    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-certs\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.131807 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.131679    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-envoy\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.131807 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.131722    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-data\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.131807 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.131779    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-credential-socket\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.132130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.131870    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-token\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.132130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.131942    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-podinfo\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.132130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.131983    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istiod-ca-cert\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.132130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.132046    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mkl9\" (UniqueName: \"kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-kube-api-access-5mkl9\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.132130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.132086    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-certs\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.132130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.132086    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-socket\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.132431 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.132174    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-data\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.132653 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.132631    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istiod-ca-cert\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.132905 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.132871    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-credential-socket\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.134846 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.134817    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-envoy\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.135194 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.135173    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-podinfo\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.193597 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.193522    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mkl9\" (UniqueName: \"kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-kube-api-access-5mkl9\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.193597 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.193574    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-token\") pod \"e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") " pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.225986 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.225962    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:15.654231 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.654205    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"]
Oct 30 19:52:15.656152 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:52:15.656128    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac7b54c4_3e25_4f25_8119_dcc6ee358eb2.slice/crio-3583f8be29455c1153658805c341ad101cc8f06eeb057811287a21af66fbad6e WatchSource:0}: Error finding container 3583f8be29455c1153658805c341ad101cc8f06eeb057811287a21af66fbad6e: Status 404 returned error can't find the container with id 3583f8be29455c1153658805c341ad101cc8f06eeb057811287a21af66fbad6e
Oct 30 19:52:15.658557 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.658523    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:15.658636 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.658606    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:15.658677 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:15.658636    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:52:16.325613 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:16.325578    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:16.517205 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:16.517169    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz" event={"ID":"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2","Type":"ContainerStarted","Data":"abf0d217f8f6ef9b0f104d58d4f95e457ac58a96bfa57ad6736bc0484d6c258b"}
Oct 30 19:52:16.517382 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:16.517228    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz" event={"ID":"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2","Type":"ContainerStarted","Data":"3583f8be29455c1153658805c341ad101cc8f06eeb057811287a21af66fbad6e"}
Oct 30 19:52:17.237427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:17.237391    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:17.244214 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:17.244174    5704 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused" start-of-body=
Oct 30 19:52:17.244410 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:17.244274    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz" podUID="ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused"
Oct 30 19:52:18.235443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:18.235405    5704 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused" start-of-body=
Oct 30 19:52:18.236066 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:18.236016    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz" podUID="ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused"
Oct 30 19:52:18.373276 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:18.373231    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:18.789628 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:18.789590    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:18.979482 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:18.979451    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:19.166841 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:19.166735    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:19.241455 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:19.241420    5704 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused" start-of-body=
Oct 30 19:52:19.242066 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:19.242008    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz" podUID="ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused"
Oct 30 19:52:20.232420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:20.232383    5704 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused" start-of-body=
Oct 30 19:52:20.232707 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:20.232677    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz" podUID="ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused"
Oct 30 19:52:21.241592 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:21.241557    5704 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused" start-of-body=
Oct 30 19:52:21.242210 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:21.242180    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz" podUID="ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.168:15021/healthz/ready\": dial tcp 10.134.0.168:15021: connect: connection refused"
Oct 30 19:52:22.231147 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:22.231119    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:22.231582 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:22.231567    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:22.232205 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:22.232183    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:52:22.396173 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:22.396114    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz" podStartSLOduration=8.39610144 podStartE2EDuration="8.39610144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:52:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:52:16.638444324 +0000 UTC m=+3909.584612226" watchObservedRunningTime="2026-10-30 19:52:22.39610144 +0000 UTC m=+3915.342269343"
Oct 30 19:52:23.382184 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:23.382153    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:23.610628 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:23.610595    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:24.946709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:24.946670    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:25.641989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:25.641952    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:25.869940 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:25.869887    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:28.370475 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:28.370430    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:29.538392 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:29.538361    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:29.783323 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:29.783293    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:32.640730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:32.640698    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:32.863328 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:32.863297    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:33.384099 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:52:33.384065    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:52:33.560397 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:33.560360    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:38.167722 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:38.167687    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:41.381047 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:41.381001    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:52:42.769316 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:42.769280    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:47.402250 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:47.402211    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:52.035726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:52.035692    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:52.234068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:52.230123    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:56.901515 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:56.901464    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:59.613050 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:59.612992    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:52:59.794530 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:52:59.794495    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:01.499104 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:01.496722    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:03.384009 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:53:03.383971    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:53:06.153195 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:06.153160    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:10.968365 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:10.968324    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:15.559516 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:15.559474    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:16.567204 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:16.567159    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:20.241961 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:20.241773    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:24.993583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:24.993525    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:28.171359 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:28.171327    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:30.342636 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:30.339779    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:31.441995 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:31.441959    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:33.384335 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:53:33.384303    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:53:35.276093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:35.276024    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:40.348476 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:40.348441    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:44.424882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:44.424842    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:53:45.311283 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:45.311235    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:49.883415 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:49.883385    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"]
Oct 30 19:53:49.887376 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:49.887354    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:49.936057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:49.936005    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-7b6aee-serving-cert\""
Oct 30 19:53:49.959287 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:49.959257    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/622d8410-59c1-4a09-acce-2aae15528bac-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:49.959417 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:49.959307    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-metrics-certs\") pod \"maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:49.959417 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:49.959388    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hhx8\" (UniqueName: \"kubernetes.io/projected/622d8410-59c1-4a09-acce-2aae15528bac-kube-api-access-6hhx8\") pod \"maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:49.959504 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:49.959417    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-maas-api-tls\") pod \"maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:49.990629 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:49.990606    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"]
Oct 30 19:53:50.060997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:50.060968    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/622d8410-59c1-4a09-acce-2aae15528bac-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:50.061190 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:50.061016    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-metrics-certs\") pod \"maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:50.061190 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:50.061075    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hhx8\" (UniqueName: \"kubernetes.io/projected/622d8410-59c1-4a09-acce-2aae15528bac-kube-api-access-6hhx8\") pod \"maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:50.061190 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:50.061130    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-maas-api-tls\") pod \"maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:50.061392 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:53:50.061365    5704 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-serving-cert: secret "maas-api-e2e-extmod-a-7b6aee-serving-cert" not found
Oct 30 19:53:50.061523 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:53:50.061456    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-maas-api-tls podName:622d8410-59c1-4a09-acce-2aae15528bac nodeName:}" failed. No retries permitted until 2026-10-30 19:53:50.561434489 +0000 UTC m=+4003.507602370 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-maas-api-tls") pod "maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr" (UID: "622d8410-59c1-4a09-acce-2aae15528bac") : secret "maas-api-e2e-extmod-a-7b6aee-serving-cert" not found
Oct 30 19:53:50.061722 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:50.061706    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/622d8410-59c1-4a09-acce-2aae15528bac-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:50.063490 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:50.063469    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-metrics-certs\") pod \"maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:50.118741 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:50.118713    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:50.119187 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:50.119165    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hhx8\" (UniqueName: \"kubernetes.io/projected/622d8410-59c1-4a09-acce-2aae15528bac-kube-api-access-6hhx8\") pod \"maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:50.397386 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:50.397348    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:50.567705 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:50.567667    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-maas-api-tls\") pod \"maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:50.570020 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:50.569991    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-maas-api-tls\") pod \"maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:50.801302 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:50.801275    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:51.176518 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:53:51.176486    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod622d8410_59c1_4a09_acce_2aae15528bac.slice/crio-4c7deb3423c45928d4cd44a12661f1315ff88208c97b8c76e5929307517f0d9f WatchSource:0}: Error finding container 4c7deb3423c45928d4cd44a12661f1315ff88208c97b8c76e5929307517f0d9f: Status 404 returned error can't find the container with id 4c7deb3423c45928d4cd44a12661f1315ff88208c97b8c76e5929307517f0d9f
Oct 30 19:53:51.185776 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:51.181562    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"]
Oct 30 19:53:52.179028 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:52.178997    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr" event={"ID":"622d8410-59c1-4a09-acce-2aae15528bac","Type":"ContainerStarted","Data":"eaa36714e478ab47ed50e55b7fa34dce64db7a60fd35ef383fcc4c20c791cb82"}
Oct 30 19:53:52.179442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:52.179082    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:52.179442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:52.179107    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr" event={"ID":"622d8410-59c1-4a09-acce-2aae15528bac","Type":"ContainerStarted","Data":"4c7deb3423c45928d4cd44a12661f1315ff88208c97b8c76e5929307517f0d9f"}
Oct 30 19:53:52.302872 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:52.302824    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr" podStartSLOduration=2.412470705 podStartE2EDuration="3.302810062s" totalImagesPullingTime="890.339357ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:53:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:53:52.301268954 +0000 UTC m=+4005.247436858" watchObservedRunningTime="2026-10-30 19:53:52.302810062 +0000 UTC m=+4005.248977964"
Oct 30 19:53:53.816989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:53.816952    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:54.007017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:54.006981    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:54.878629 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:54.878601    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:53:58.192234 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:58.192198    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:53:59.479103 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:53:59.479058    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:03.384936 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:54:03.384901    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:54:04.007516 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:04.007480    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:04.198079 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:04.198003    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:04.382870 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:04.382789    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:08.876359 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:08.876327    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:13.744251 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:13.740464    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:16.901183 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:16.901147    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:18.682401 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:18.682367    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:22.833758 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:22.833723    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"]
Oct 30 19:54:22.933576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:22.933541    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"]
Oct 30 19:54:23.213508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:23.213426    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:23.416346 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:23.416056    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:27.834406 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:27.834365    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt" podUID="f26b13a2-df3c-4e26-a7af-0bd40ddd3190" containerName="payload-processing" containerID="cri-o://91b12a9707c0dfe322d78dacd58555470d3d3924f4afd8e2be46cc03ee00f6ce" gracePeriod=25
Oct 30 19:54:27.934131 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:27.934083    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6" podUID="185e2fc0-40ef-4f92-87fa-d2257741f604" containerName="payload-pre-processing" containerID="cri-o://e41b14b826fcba55c156d0f5318002c8b5b5e032375ba8d5d4355c5c3b6b5d29" gracePeriod=25
Oct 30 19:54:28.106292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.106231    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"
Oct 30 19:54:28.211195 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.211174    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"
Oct 30 19:54:28.273745 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.273719    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-kube-api-access-5txxs\" (UniqueName: \"kubernetes.io/projected/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-kube-api-access-5txxs\") pod \"f26b13a2-df3c-4e26-a7af-0bd40ddd3190\" (UID: \"f26b13a2-df3c-4e26-a7af-0bd40ddd3190\") "
Oct 30 19:54:28.273928 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.273914    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-plugins-config-volume\") pod \"f26b13a2-df3c-4e26-a7af-0bd40ddd3190\" (UID: \"f26b13a2-df3c-4e26-a7af-0bd40ddd3190\") "
Oct 30 19:54:28.274235 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.274212    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-plugins-config-volume" pod "f26b13a2-df3c-4e26-a7af-0bd40ddd3190" (UID: "f26b13a2-df3c-4e26-a7af-0bd40ddd3190"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:54:28.274419 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.274401    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:28.275726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.275706    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-kube-api-access-5txxs" pod "f26b13a2-df3c-4e26-a7af-0bd40ddd3190" (UID: "f26b13a2-df3c-4e26-a7af-0bd40ddd3190"). InnerVolumeSpecName "kube-api-access-5txxs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:28.377583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.375857    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/185e2fc0-40ef-4f92-87fa-d2257741f604-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/185e2fc0-40ef-4f92-87fa-d2257741f604-plugins-config-volume\") pod \"185e2fc0-40ef-4f92-87fa-d2257741f604\" (UID: \"185e2fc0-40ef-4f92-87fa-d2257741f604\") "
Oct 30 19:54:28.377583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.375955    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/185e2fc0-40ef-4f92-87fa-d2257741f604-kube-api-access-mxzlv\" (UniqueName: \"kubernetes.io/projected/185e2fc0-40ef-4f92-87fa-d2257741f604-kube-api-access-mxzlv\") pod \"185e2fc0-40ef-4f92-87fa-d2257741f604\" (UID: \"185e2fc0-40ef-4f92-87fa-d2257741f604\") "
Oct 30 19:54:28.377583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.376589    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5txxs\" (UniqueName: \"kubernetes.io/projected/f26b13a2-df3c-4e26-a7af-0bd40ddd3190-kube-api-access-5txxs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:28.377583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.377394    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/185e2fc0-40ef-4f92-87fa-d2257741f604-plugins-config-volume" pod "185e2fc0-40ef-4f92-87fa-d2257741f604" (UID: "185e2fc0-40ef-4f92-87fa-d2257741f604"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:54:28.384073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.379684    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:28.397856 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.397796    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/185e2fc0-40ef-4f92-87fa-d2257741f604-kube-api-access-mxzlv" pod "185e2fc0-40ef-4f92-87fa-d2257741f604" (UID: "185e2fc0-40ef-4f92-87fa-d2257741f604"). InnerVolumeSpecName "kube-api-access-mxzlv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:28.409913 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.409883    5704 generic.go:356] "Generic (PLEG): container finished" podID="185e2fc0-40ef-4f92-87fa-d2257741f604" containerID="e41b14b826fcba55c156d0f5318002c8b5b5e032375ba8d5d4355c5c3b6b5d29" exitCode=0
Oct 30 19:54:28.410105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.409932    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"
Oct 30 19:54:28.410105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.409979    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6" event={"ID":"185e2fc0-40ef-4f92-87fa-d2257741f604","Type":"ContainerDied","Data":"e41b14b826fcba55c156d0f5318002c8b5b5e032375ba8d5d4355c5c3b6b5d29"}
Oct 30 19:54:28.410105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.410053    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6" event={"ID":"185e2fc0-40ef-4f92-87fa-d2257741f604","Type":"ContainerDied","Data":"b291d979dc453e6df1b2b7e3593a5dddadd72a87b630b41825144ad7850c403f"}
Oct 30 19:54:28.410105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.410084    5704 scope.go:122] "RemoveContainer" containerID="e41b14b826fcba55c156d0f5318002c8b5b5e032375ba8d5d4355c5c3b6b5d29"
Oct 30 19:54:28.411749 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.411721    5704 generic.go:356] "Generic (PLEG): container finished" podID="f26b13a2-df3c-4e26-a7af-0bd40ddd3190" containerID="91b12a9707c0dfe322d78dacd58555470d3d3924f4afd8e2be46cc03ee00f6ce" exitCode=0
Oct 30 19:54:28.411854 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.411764    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"
Oct 30 19:54:28.411854 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.411786    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt" event={"ID":"f26b13a2-df3c-4e26-a7af-0bd40ddd3190","Type":"ContainerDied","Data":"91b12a9707c0dfe322d78dacd58555470d3d3924f4afd8e2be46cc03ee00f6ce"}
Oct 30 19:54:28.411854 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.411848    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt" event={"ID":"f26b13a2-df3c-4e26-a7af-0bd40ddd3190","Type":"ContainerDied","Data":"530c7cf828afe0babf04ac8502ff263f204beab9a102773cd94bf93fdecf68d3"}
Oct 30 19:54:28.427483 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.426059    5704 scope.go:122] "RemoveContainer" containerID="e41b14b826fcba55c156d0f5318002c8b5b5e032375ba8d5d4355c5c3b6b5d29"
Oct 30 19:54:28.427759 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:54:28.427736    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e41b14b826fcba55c156d0f5318002c8b5b5e032375ba8d5d4355c5c3b6b5d29\": container with ID starting with e41b14b826fcba55c156d0f5318002c8b5b5e032375ba8d5d4355c5c3b6b5d29 not found: ID does not exist" containerID="e41b14b826fcba55c156d0f5318002c8b5b5e032375ba8d5d4355c5c3b6b5d29"
Oct 30 19:54:28.427837 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.427773    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e41b14b826fcba55c156d0f5318002c8b5b5e032375ba8d5d4355c5c3b6b5d29"} err="failed to get container status \"e41b14b826fcba55c156d0f5318002c8b5b5e032375ba8d5d4355c5c3b6b5d29\": rpc error: code = NotFound desc = could not find container \"e41b14b826fcba55c156d0f5318002c8b5b5e032375ba8d5d4355c5c3b6b5d29\": container with ID starting with e41b14b826fcba55c156d0f5318002c8b5b5e032375ba8d5d4355c5c3b6b5d29 not found: ID does not exist"
Oct 30 19:54:28.427837 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.427797    5704 scope.go:122] "RemoveContainer" containerID="91b12a9707c0dfe322d78dacd58555470d3d3924f4afd8e2be46cc03ee00f6ce"
Oct 30 19:54:28.437013 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.436994    5704 scope.go:122] "RemoveContainer" containerID="91b12a9707c0dfe322d78dacd58555470d3d3924f4afd8e2be46cc03ee00f6ce"
Oct 30 19:54:28.437285 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:54:28.437267    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91b12a9707c0dfe322d78dacd58555470d3d3924f4afd8e2be46cc03ee00f6ce\": container with ID starting with 91b12a9707c0dfe322d78dacd58555470d3d3924f4afd8e2be46cc03ee00f6ce not found: ID does not exist" containerID="91b12a9707c0dfe322d78dacd58555470d3d3924f4afd8e2be46cc03ee00f6ce"
Oct 30 19:54:28.437347 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.437293    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91b12a9707c0dfe322d78dacd58555470d3d3924f4afd8e2be46cc03ee00f6ce"} err="failed to get container status \"91b12a9707c0dfe322d78dacd58555470d3d3924f4afd8e2be46cc03ee00f6ce\": rpc error: code = NotFound desc = could not find container \"91b12a9707c0dfe322d78dacd58555470d3d3924f4afd8e2be46cc03ee00f6ce\": container with ID starting with 91b12a9707c0dfe322d78dacd58555470d3d3924f4afd8e2be46cc03ee00f6ce not found: ID does not exist"
Oct 30 19:54:28.477594 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.477573    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/185e2fc0-40ef-4f92-87fa-d2257741f604-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:28.477594 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.477594    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxzlv\" (UniqueName: \"kubernetes.io/projected/185e2fc0-40ef-4f92-87fa-d2257741f604-kube-api-access-mxzlv\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:28.564962 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.564934    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"]
Oct 30 19:54:28.568760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.568733    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-b330be-f99498799-dvfz6"]
Oct 30 19:54:28.670092 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.669999    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"]
Oct 30 19:54:28.674479 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:28.674453    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-b330be-56fc6dd49b-t7qnt"]
Oct 30 19:54:29.180176 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:29.180145    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="185e2fc0-40ef-4f92-87fa-d2257741f604" path="/var/lib/kubelet/pods/185e2fc0-40ef-4f92-87fa-d2257741f604/volumes"
Oct 30 19:54:29.180590 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:29.180576    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f26b13a2-df3c-4e26-a7af-0bd40ddd3190" path="/var/lib/kubelet/pods/f26b13a2-df3c-4e26-a7af-0bd40ddd3190/volumes"
Oct 30 19:54:29.820880 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:29.820832    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:30.001406 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.001375    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"]
Oct 30 19:54:30.002059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.002015    5704 cpu_manager.go:414] "containerMap: removing container" podUID="185e2fc0-40ef-4f92-87fa-d2257741f604" containerName="payload-pre-processing"
Oct 30 19:54:30.002059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.002042    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="185e2fc0-40ef-4f92-87fa-d2257741f604" containerName="payload-pre-processing"
Oct 30 19:54:30.002059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.002058    5704 cpu_manager.go:414] "containerMap: removing container" podUID="f26b13a2-df3c-4e26-a7af-0bd40ddd3190" containerName="payload-processing"
Oct 30 19:54:30.002271 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.002063    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f26b13a2-df3c-4e26-a7af-0bd40ddd3190" containerName="payload-processing"
Oct 30 19:54:30.002271 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.002163    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="185e2fc0-40ef-4f92-87fa-d2257741f604" containerName="payload-pre-processing"
Oct 30 19:54:30.002271 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.002171    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="f26b13a2-df3c-4e26-a7af-0bd40ddd3190" containerName="payload-processing"
Oct 30 19:54:30.007526 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.007504    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.057723 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.057691    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-a-7b6aee-openshift-default-dockercfg-n2tpq\""
Oct 30 19:54:30.106268 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.106195    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"]
Oct 30 19:54:30.195569 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.195538    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istiod-ca-cert\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.196068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.195591    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-credential-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.196068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.195715    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-certs\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.196068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.195762    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-data\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.196068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.195846    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.196068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.195914    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-token\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.196068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.195940    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-envoy\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.196068 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.195970    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-podinfo\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.196375 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.196123    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppgkf\" (UniqueName: \"kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-kube-api-access-ppgkf\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.296930 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.296900    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-envoy\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.297105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.296942    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-podinfo\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.297105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.296999    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppgkf\" (UniqueName: \"kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-kube-api-access-ppgkf\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.297105 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.297083    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istiod-ca-cert\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.297307 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.297123    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-credential-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.297307 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.297173    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-certs\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.297307 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.297205    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-data\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.297307 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.297231    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.297612 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.297583    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-credential-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.297612 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.297605    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-token\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.297823 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.297651    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-certs\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.297823 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.297789    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-data\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.297940 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.297877    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.297940 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.297881    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istiod-ca-cert\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.299333 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.299313    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-envoy\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.299491 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.299473    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-podinfo\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.357420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.357361    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppgkf\" (UniqueName: \"kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-kube-api-access-ppgkf\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.357613 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.357589    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-token\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:30.621378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:30.621305    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:31.042627 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:31.042498    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"]
Oct 30 19:54:31.045442 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:54:31.045417    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa6c415c_9fee_4b1e_979f_5764d5cba9a4.slice/crio-11c548d641b71fbad8cc8f92a5889b45b24611f584f2bde7400a904e0a71d167 WatchSource:0}: Error finding container 11c548d641b71fbad8cc8f92a5889b45b24611f584f2bde7400a904e0a71d167: Status 404 returned error can't find the container with id 11c548d641b71fbad8cc8f92a5889b45b24611f584f2bde7400a904e0a71d167
Oct 30 19:54:31.048224 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:31.048187    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:54:31.048336 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:31.048269    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:54:31.048336 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:31.048317    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:54:31.435186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:31.434516    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn" event={"ID":"fa6c415c-9fee-4b1e-979f-5764d5cba9a4","Type":"ContainerStarted","Data":"f7ebb944bd6dc11e183dd72e7cfc2c178bd88aef147906596d800603ed08b442"}
Oct 30 19:54:31.435186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:31.434581    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn" event={"ID":"fa6c415c-9fee-4b1e-979f-5764d5cba9a4","Type":"ContainerStarted","Data":"11c548d641b71fbad8cc8f92a5889b45b24611f584f2bde7400a904e0a71d167"}
Oct 30 19:54:31.621906 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:31.621880    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:31.641644 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:31.641593    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn" podStartSLOduration=2.641579434 podStartE2EDuration="2.641579434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:54:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:54:31.593859346 +0000 UTC m=+4044.540027248" watchObservedRunningTime="2026-10-30 19:54:31.641579434 +0000 UTC m=+4044.587747403"
Oct 30 19:54:32.644215 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:32.644158    5704 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.170:15021/healthz/ready\": dial tcp 10.134.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:54:32.644752 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:32.644232    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn" podUID="fa6c415c-9fee-4b1e-979f-5764d5cba9a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.170:15021/healthz/ready\": dial tcp 10.134.0.170:15021: connect: connection refused"
Oct 30 19:54:33.159062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:33.156430    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:33.384487 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:54:33.384452    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:54:33.626058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:33.625384    5704 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.170:15021/healthz/ready\": dial tcp 10.134.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:54:33.626058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:33.625440    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn" podUID="fa6c415c-9fee-4b1e-979f-5764d5cba9a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.170:15021/healthz/ready\": dial tcp 10.134.0.170:15021: connect: connection refused"
Oct 30 19:54:34.644060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:34.640430    5704 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.170:15021/healthz/ready\": dial tcp 10.134.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:54:34.644060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:34.640505    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn" podUID="fa6c415c-9fee-4b1e-979f-5764d5cba9a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.170:15021/healthz/ready\": dial tcp 10.134.0.170:15021: connect: connection refused"
Oct 30 19:54:35.636064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:35.635412    5704 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.170:15021/healthz/ready\": dial tcp 10.134.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:54:35.636064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:35.635505    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn" podUID="fa6c415c-9fee-4b1e-979f-5764d5cba9a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.170:15021/healthz/ready\": dial tcp 10.134.0.170:15021: connect: connection refused"
Oct 30 19:54:36.632057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:36.628362    5704 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.170:15021/healthz/ready\": dial tcp 10.134.0.170:15021: connect: connection refused" start-of-body=
Oct 30 19:54:36.632057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:36.628424    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn" podUID="fa6c415c-9fee-4b1e-979f-5764d5cba9a4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.170:15021/healthz/ready\": dial tcp 10.134.0.170:15021: connect: connection refused"
Oct 30 19:54:37.627169 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:37.627138    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:37.627518 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:37.627498    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:37.628127 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:37.628110    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:54:37.834911 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:37.834879    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"]
Oct 30 19:54:37.923017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:37.922931    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"]
Oct 30 19:54:38.022410 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:38.022368    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:38.208579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:38.208427    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:38.391088 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:38.391051    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:42.704335 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:42.704298    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:42.836147 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:42.836106    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll" podUID="7dd417f3-3db2-401a-abdf-15fff44fc711" containerName="payload-processing" containerID="cri-o://a3b16677194275b337043f55d1249c6c71395e81dae671d6ad5acb853e155939" gracePeriod=25
Oct 30 19:54:42.923651 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:42.923619    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f" podUID="6123ea10-1fe2-482b-9f1d-e6523f98cd7b" containerName="payload-pre-processing" containerID="cri-o://2845e97cb587fde1a1b464d10fc9f231c2ab177ddc75535219ed4fb3e8f6b08b" gracePeriod=25
Oct 30 19:54:43.197203 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.197178    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"
Oct 30 19:54:43.232579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.232558    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"
Oct 30 19:54:43.290423 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.290366    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-kube-api-access-t5rw5\" (UniqueName: \"kubernetes.io/projected/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-kube-api-access-t5rw5\") pod \"6123ea10-1fe2-482b-9f1d-e6523f98cd7b\" (UID: \"6123ea10-1fe2-482b-9f1d-e6523f98cd7b\") "
Oct 30 19:54:43.290542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.290440    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-plugins-config-volume\") pod \"6123ea10-1fe2-482b-9f1d-e6523f98cd7b\" (UID: \"6123ea10-1fe2-482b-9f1d-e6523f98cd7b\") "
Oct 30 19:54:43.290615 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.290549    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7dd417f3-3db2-401a-abdf-15fff44fc711-kube-api-access-qpqcm\" (UniqueName: \"kubernetes.io/projected/7dd417f3-3db2-401a-abdf-15fff44fc711-kube-api-access-qpqcm\") pod \"7dd417f3-3db2-401a-abdf-15fff44fc711\" (UID: \"7dd417f3-3db2-401a-abdf-15fff44fc711\") "
Oct 30 19:54:43.290615 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.290597    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7dd417f3-3db2-401a-abdf-15fff44fc711-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dd417f3-3db2-401a-abdf-15fff44fc711-plugins-config-volume\") pod \"7dd417f3-3db2-401a-abdf-15fff44fc711\" (UID: \"7dd417f3-3db2-401a-abdf-15fff44fc711\") "
Oct 30 19:54:43.290737 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.290722    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-plugins-config-volume" pod "6123ea10-1fe2-482b-9f1d-e6523f98cd7b" (UID: "6123ea10-1fe2-482b-9f1d-e6523f98cd7b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:54:43.291006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.290981    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dd417f3-3db2-401a-abdf-15fff44fc711-plugins-config-volume" pod "7dd417f3-3db2-401a-abdf-15fff44fc711" (UID: "7dd417f3-3db2-401a-abdf-15fff44fc711"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:54:43.291604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.291584    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:43.291604 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.291603    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7dd417f3-3db2-401a-abdf-15fff44fc711-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:43.292423 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.292408    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-kube-api-access-t5rw5" pod "6123ea10-1fe2-482b-9f1d-e6523f98cd7b" (UID: "6123ea10-1fe2-482b-9f1d-e6523f98cd7b"). InnerVolumeSpecName "kube-api-access-t5rw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:43.292552 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.292535    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dd417f3-3db2-401a-abdf-15fff44fc711-kube-api-access-qpqcm" pod "7dd417f3-3db2-401a-abdf-15fff44fc711" (UID: "7dd417f3-3db2-401a-abdf-15fff44fc711"). InnerVolumeSpecName "kube-api-access-qpqcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:54:43.392742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.392713    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpqcm\" (UniqueName: \"kubernetes.io/projected/7dd417f3-3db2-401a-abdf-15fff44fc711-kube-api-access-qpqcm\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:43.392742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.392738    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5rw5\" (UniqueName: \"kubernetes.io/projected/6123ea10-1fe2-482b-9f1d-e6523f98cd7b-kube-api-access-t5rw5\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:54:43.543956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.543873    5704 generic.go:356] "Generic (PLEG): container finished" podID="7dd417f3-3db2-401a-abdf-15fff44fc711" containerID="a3b16677194275b337043f55d1249c6c71395e81dae671d6ad5acb853e155939" exitCode=0
Oct 30 19:54:43.543956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.543912    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"
Oct 30 19:54:43.544197 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.544003    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll" event={"ID":"7dd417f3-3db2-401a-abdf-15fff44fc711","Type":"ContainerDied","Data":"a3b16677194275b337043f55d1249c6c71395e81dae671d6ad5acb853e155939"}
Oct 30 19:54:43.544197 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.544087    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll" event={"ID":"7dd417f3-3db2-401a-abdf-15fff44fc711","Type":"ContainerDied","Data":"9d6c56b4597777ec2b93eeca45a402f1f01c2457960dc77118e36a52c64ff6d9"}
Oct 30 19:54:43.544197 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.544112    5704 scope.go:122] "RemoveContainer" containerID="a3b16677194275b337043f55d1249c6c71395e81dae671d6ad5acb853e155939"
Oct 30 19:54:43.545370 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.545350    5704 generic.go:356] "Generic (PLEG): container finished" podID="6123ea10-1fe2-482b-9f1d-e6523f98cd7b" containerID="2845e97cb587fde1a1b464d10fc9f231c2ab177ddc75535219ed4fb3e8f6b08b" exitCode=0
Oct 30 19:54:43.545503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.545392    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"
Oct 30 19:54:43.545503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.545434    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f" event={"ID":"6123ea10-1fe2-482b-9f1d-e6523f98cd7b","Type":"ContainerDied","Data":"2845e97cb587fde1a1b464d10fc9f231c2ab177ddc75535219ed4fb3e8f6b08b"}
Oct 30 19:54:43.545503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.545474    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f" event={"ID":"6123ea10-1fe2-482b-9f1d-e6523f98cd7b","Type":"ContainerDied","Data":"a456dc7b7d54a9a8021b28211ea404096c7a1eeecb25ef4b1712d8d46d1164f4"}
Oct 30 19:54:43.558884 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.558658    5704 scope.go:122] "RemoveContainer" containerID="a3b16677194275b337043f55d1249c6c71395e81dae671d6ad5acb853e155939"
Oct 30 19:54:43.559124 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:54:43.559104    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3b16677194275b337043f55d1249c6c71395e81dae671d6ad5acb853e155939\": container with ID starting with a3b16677194275b337043f55d1249c6c71395e81dae671d6ad5acb853e155939 not found: ID does not exist" containerID="a3b16677194275b337043f55d1249c6c71395e81dae671d6ad5acb853e155939"
Oct 30 19:54:43.559182 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.559132    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3b16677194275b337043f55d1249c6c71395e81dae671d6ad5acb853e155939"} err="failed to get container status \"a3b16677194275b337043f55d1249c6c71395e81dae671d6ad5acb853e155939\": rpc error: code = NotFound desc = could not find container \"a3b16677194275b337043f55d1249c6c71395e81dae671d6ad5acb853e155939\": container with ID starting with a3b16677194275b337043f55d1249c6c71395e81dae671d6ad5acb853e155939 not found: ID does not exist"
Oct 30 19:54:43.559182 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.559150    5704 scope.go:122] "RemoveContainer" containerID="2845e97cb587fde1a1b464d10fc9f231c2ab177ddc75535219ed4fb3e8f6b08b"
Oct 30 19:54:43.568236 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.568208    5704 scope.go:122] "RemoveContainer" containerID="2845e97cb587fde1a1b464d10fc9f231c2ab177ddc75535219ed4fb3e8f6b08b"
Oct 30 19:54:43.568509 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:54:43.568488    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2845e97cb587fde1a1b464d10fc9f231c2ab177ddc75535219ed4fb3e8f6b08b\": container with ID starting with 2845e97cb587fde1a1b464d10fc9f231c2ab177ddc75535219ed4fb3e8f6b08b not found: ID does not exist" containerID="2845e97cb587fde1a1b464d10fc9f231c2ab177ddc75535219ed4fb3e8f6b08b"
Oct 30 19:54:43.568581 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.568519    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2845e97cb587fde1a1b464d10fc9f231c2ab177ddc75535219ed4fb3e8f6b08b"} err="failed to get container status \"2845e97cb587fde1a1b464d10fc9f231c2ab177ddc75535219ed4fb3e8f6b08b\": rpc error: code = NotFound desc = could not find container \"2845e97cb587fde1a1b464d10fc9f231c2ab177ddc75535219ed4fb3e8f6b08b\": container with ID starting with 2845e97cb587fde1a1b464d10fc9f231c2ab177ddc75535219ed4fb3e8f6b08b not found: ID does not exist"
Oct 30 19:54:43.698251 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.698226    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"]
Oct 30 19:54:43.703852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.703822    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-a-905752-74d5c89dc5-xq95f"]
Oct 30 19:54:43.807042 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.806964    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"]
Oct 30 19:54:43.810904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:43.810884    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-a-905752-66c9bf4df6-ghmll"]
Oct 30 19:54:45.181442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:45.181401    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6123ea10-1fe2-482b-9f1d-e6523f98cd7b" path="/var/lib/kubelet/pods/6123ea10-1fe2-482b-9f1d-e6523f98cd7b/volumes"
Oct 30 19:54:45.182002 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:45.181981    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7dd417f3-3db2-401a-abdf-15fff44fc711" path="/var/lib/kubelet/pods/7dd417f3-3db2-401a-abdf-15fff44fc711/volumes"
Oct 30 19:54:47.316328 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:47.315753    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:51.869199 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:51.869121    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:55.894987 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:55.894946    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:56.993910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:56.993876    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:57.324271 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:57.324233    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:57.621217 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:57.621117    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:58.361431 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:58.361397    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:58.589160 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:58.589126    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:58.772268 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:58.772235    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:54:59.062158 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:59.062060    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:54:59.391599 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:54:59.391510    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:03.384009 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:55:03.383973    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:55:03.519149 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:03.519120    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:04.593672 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:04.593638    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:04.770536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:04.770499    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:04.953521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:04.953439    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:06.644597 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:06.644560    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:06.824216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:06.824179    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:06.924709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:06.924631    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"]
Oct 30 19:55:06.924935 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:06.924912    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz" podUID="ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" containerName="istio-proxy" containerID="cri-o://abf0d217f8f6ef9b0f104d58d4f95e457ac58a96bfa57ad6736bc0484d6c258b" gracePeriod=30
Oct 30 19:55:07.028054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:07.024515    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:07.234186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:07.234108    5704 patch_prober.go:29] interesting pod/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:55:07.234325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:07.234183    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz" podUID="ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:55:07.276359 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:07.276333    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:08.505205 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:08.505171    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:08.929007 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:08.928971    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:09.520870 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:09.520816    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:10.002694 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:10.002659    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:12.201219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.201146    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:55:12.389730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.389699    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-credential-socket\") pod \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") "
Oct 30 19:55:12.389877 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.389786    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-socket\") pod \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") "
Oct 30 19:55:12.389877 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.389816    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-token\" (UniqueName: \"kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-token\") pod \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") "
Oct 30 19:55:12.389877 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.389837    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-podinfo\") pod \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") "
Oct 30 19:55:12.389877 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.389855    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-data\") pod \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") "
Oct 30 19:55:12.390095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.389928    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-kube-api-access-5mkl9\" (UniqueName: \"kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-kube-api-access-5mkl9\") pod \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") "
Oct 30 19:55:12.390095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.389945    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-certs\") pod \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") "
Oct 30 19:55:12.390095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.389973    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istiod-ca-cert\") pod \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") "
Oct 30 19:55:12.390095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.389995    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-credential-socket" pod "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" (UID: "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:12.390095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.390025    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-envoy\") pod \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\" (UID: \"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2\") "
Oct 30 19:55:12.390369 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.390105    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-socket" pod "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" (UID: "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:12.390369 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.390278    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-certs" pod "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" (UID: "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:12.390369 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.390295    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-data" pod "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" (UID: "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:12.390542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.390366    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istiod-ca-cert" pod "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" (UID: "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:55:12.390630 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.390613    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.390697 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.390631    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.390697 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.390641    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.390697 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.390650    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.390697 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.390660    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.392900 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.392866    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-envoy" pod "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" (UID: "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:55:12.393354 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.393327    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-token" pod "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" (UID: "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:55:12.393474 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.393455    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-podinfo" pod "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" (UID: "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:55:12.393537 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.393515    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-kube-api-access-5mkl9" pod "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" (UID: "ac7b54c4-3e25-4f25-8119-dcc6ee358eb2"). InnerVolumeSpecName "kube-api-access-5mkl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:55:12.492300 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.492220    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mkl9\" (UniqueName: \"kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-kube-api-access-5mkl9\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.492300 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.492257    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.492300 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.492271    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.492300 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.492286    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:12.729893 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.729861    5704 generic.go:356] "Generic (PLEG): container finished" podID="ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" containerID="abf0d217f8f6ef9b0f104d58d4f95e457ac58a96bfa57ad6736bc0484d6c258b" exitCode=0
Oct 30 19:55:12.730072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.729900    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz" event={"ID":"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2","Type":"ContainerDied","Data":"abf0d217f8f6ef9b0f104d58d4f95e457ac58a96bfa57ad6736bc0484d6c258b"}
Oct 30 19:55:12.730072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.729911    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"
Oct 30 19:55:12.730072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.729938    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz" event={"ID":"ac7b54c4-3e25-4f25-8119-dcc6ee358eb2","Type":"ContainerDied","Data":"3583f8be29455c1153658805c341ad101cc8f06eeb057811287a21af66fbad6e"}
Oct 30 19:55:12.730072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.729960    5704 scope.go:122] "RemoveContainer" containerID="abf0d217f8f6ef9b0f104d58d4f95e457ac58a96bfa57ad6736bc0484d6c258b"
Oct 30 19:55:12.740693 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.740678    5704 scope.go:122] "RemoveContainer" containerID="abf0d217f8f6ef9b0f104d58d4f95e457ac58a96bfa57ad6736bc0484d6c258b"
Oct 30 19:55:12.740970 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:55:12.740951    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abf0d217f8f6ef9b0f104d58d4f95e457ac58a96bfa57ad6736bc0484d6c258b\": container with ID starting with abf0d217f8f6ef9b0f104d58d4f95e457ac58a96bfa57ad6736bc0484d6c258b not found: ID does not exist" containerID="abf0d217f8f6ef9b0f104d58d4f95e457ac58a96bfa57ad6736bc0484d6c258b"
Oct 30 19:55:12.741047 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.740978    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abf0d217f8f6ef9b0f104d58d4f95e457ac58a96bfa57ad6736bc0484d6c258b"} err="failed to get container status \"abf0d217f8f6ef9b0f104d58d4f95e457ac58a96bfa57ad6736bc0484d6c258b\": rpc error: code = NotFound desc = could not find container \"abf0d217f8f6ef9b0f104d58d4f95e457ac58a96bfa57ad6736bc0484d6c258b\": container with ID starting with abf0d217f8f6ef9b0f104d58d4f95e457ac58a96bfa57ad6736bc0484d6c258b not found: ID does not exist"
Oct 30 19:55:12.884998 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.884963    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"]
Oct 30 19:55:12.931162 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:12.931135    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-b330be-openshift-default-7d97dd6497-sdvhz"]
Oct 30 19:55:13.180374 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:13.180308    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" path="/var/lib/kubelet/pods/ac7b54c4-3e25-4f25-8119-dcc6ee358eb2/volumes"
Oct 30 19:55:13.597005 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:13.596964    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:14.191025 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:14.190988    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:14.395643 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:14.395603    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:14.585550 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:14.585516    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:15.365281 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.365249    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"]
Oct 30 19:55:15.365851 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.365837    5704 cpu_manager.go:414] "containerMap: removing container" podUID="6123ea10-1fe2-482b-9f1d-e6523f98cd7b" containerName="payload-pre-processing"
Oct 30 19:55:15.365898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.365852    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6123ea10-1fe2-482b-9f1d-e6523f98cd7b" containerName="payload-pre-processing"
Oct 30 19:55:15.365898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.365889    5704 cpu_manager.go:414] "containerMap: removing container" podUID="ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" containerName="istio-proxy"
Oct 30 19:55:15.365898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.365895    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" containerName="istio-proxy"
Oct 30 19:55:15.366004 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.365906    5704 cpu_manager.go:414] "containerMap: removing container" podUID="7dd417f3-3db2-401a-abdf-15fff44fc711" containerName="payload-processing"
Oct 30 19:55:15.366004 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.365911    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7dd417f3-3db2-401a-abdf-15fff44fc711" containerName="payload-processing"
Oct 30 19:55:15.366004 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.365994    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="7dd417f3-3db2-401a-abdf-15fff44fc711" containerName="payload-processing"
Oct 30 19:55:15.366128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.366005    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="6123ea10-1fe2-482b-9f1d-e6523f98cd7b" containerName="payload-pre-processing"
Oct 30 19:55:15.366128 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.366018    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="ac7b54c4-3e25-4f25-8119-dcc6ee358eb2" containerName="istio-proxy"
Oct 30 19:55:15.369388 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.369372    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.421707 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.421676    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-data\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.421862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.421731    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af934c90-d9b7-427d-acce-67f9cc332a82-istio-podinfo\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.421862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.421796    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-credential-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.421995 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.421897    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-istio-token\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.421995 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.421941    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-certs\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.422148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.422091    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqzt2\" (UniqueName: \"kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-kube-api-access-sqzt2\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.422195 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.422143    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.422238 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.422211    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af934c90-d9b7-427d-acce-67f9cc332a82-istiod-ca-cert\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.422278 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.422238    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-envoy\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.468699 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.468666    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"]
Oct 30 19:55:15.522842 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.522812    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af934c90-d9b7-427d-acce-67f9cc332a82-istiod-ca-cert\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.523003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.522853    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-envoy\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.523003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.522925    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-data\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.523003 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.522978    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af934c90-d9b7-427d-acce-67f9cc332a82-istio-podinfo\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.523169 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.523002    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-credential-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.523169 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.523064    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-istio-token\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.523169 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.523101    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-certs\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.523326 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.523170    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqzt2\" (UniqueName: \"kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-kube-api-access-sqzt2\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.523326 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.523210    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.523436 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.523403    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-data\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.523537 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.523461    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-credential-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.523609 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.523543    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-socket\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.523675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.523646    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af934c90-d9b7-427d-acce-67f9cc332a82-istiod-ca-cert\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.523737 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.523718    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-certs\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.525227 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.525205    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-envoy\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.525433 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.525416    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af934c90-d9b7-427d-acce-67f9cc332a82-istio-podinfo\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.580995 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.580968    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-istio-token\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.581494 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.581028    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqzt2\" (UniqueName: \"kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-kube-api-access-sqzt2\") pod \"e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") " pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:15.683137 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:15.683063    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:16.108482 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:16.108450    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"]
Oct 30 19:55:16.110699 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:55:16.110668    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf934c90_d9b7_427d_acce_67f9cc332a82.slice/crio-b74ab380f4f9a5b918a71311b81068066b8588fbeb8710702ab5c5912efcdbbf WatchSource:0}: Error finding container b74ab380f4f9a5b918a71311b81068066b8588fbeb8710702ab5c5912efcdbbf: Status 404 returned error can't find the container with id b74ab380f4f9a5b918a71311b81068066b8588fbeb8710702ab5c5912efcdbbf
Oct 30 19:55:16.113272 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:16.113239    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:55:16.113383 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:16.113317    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:55:16.113383 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:16.113360    5704 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236072Ki","pods":"250"}
Oct 30 19:55:16.761275 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:16.761236    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c" event={"ID":"af934c90-d9b7-427d-acce-67f9cc332a82","Type":"ContainerStarted","Data":"e8dff82dec3d9d865de729409d763cc5d68add06ce3ce83d094de13bcc93b43b"}
Oct 30 19:55:16.761813 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:16.761316    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c" event={"ID":"af934c90-d9b7-427d-acce-67f9cc332a82","Type":"ContainerStarted","Data":"b74ab380f4f9a5b918a71311b81068066b8588fbeb8710702ab5c5912efcdbbf"}
Oct 30 19:55:16.928270 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:16.928220    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c" podStartSLOduration=1.92820687 podStartE2EDuration="1.92820687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:55:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:55:16.924582663 +0000 UTC m=+4089.870750569" watchObservedRunningTime="2026-10-30 19:55:16.92820687 +0000 UTC m=+4089.874374773"
Oct 30 19:55:17.245913 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:17.245877    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:17.449051 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:17.449004    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:17.645156 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:17.645060    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:17.695929 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:17.695899    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:17.701290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:17.701258    5704 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.171:15021/healthz/ready\": dial tcp 10.134.0.171:15021: connect: connection refused" start-of-body=
Oct 30 19:55:17.701535 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:17.701509    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c" podUID="af934c90-d9b7-427d-acce-67f9cc332a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.171:15021/healthz/ready\": dial tcp 10.134.0.171:15021: connect: connection refused"
Oct 30 19:55:18.527209 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:18.527173    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:18.684223 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:18.684193    5704 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.171:15021/healthz/ready\": dial tcp 10.134.0.171:15021: connect: connection refused" start-of-body=
Oct 30 19:55:18.684399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:18.684239    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c" podUID="af934c90-d9b7-427d-acce-67f9cc332a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.171:15021/healthz/ready\": dial tcp 10.134.0.171:15021: connect: connection refused"
Oct 30 19:55:19.684172 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:19.684132    5704 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.171:15021/healthz/ready\": dial tcp 10.134.0.171:15021: connect: connection refused" start-of-body=
Oct 30 19:55:19.684585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:19.684219    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c" podUID="af934c90-d9b7-427d-acce-67f9cc332a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.171:15021/healthz/ready\": dial tcp 10.134.0.171:15021: connect: connection refused"
Oct 30 19:55:19.740621 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:19.740586    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"]
Oct 30 19:55:20.157454 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:20.157355    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:20.345402 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:20.345367    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:20.683478 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:20.683441    5704 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.171:15021/healthz/ready\": dial tcp 10.134.0.171:15021: connect: connection refused" start-of-body=
Oct 30 19:55:20.683651 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:20.683535    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c" podUID="af934c90-d9b7-427d-acce-67f9cc332a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.171:15021/healthz/ready\": dial tcp 10.134.0.171:15021: connect: connection refused"
Oct 30 19:55:21.683453 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:21.683400    5704 patch_prober.go:29] interesting pod/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.171:15021/healthz/ready\": dial tcp 10.134.0.171:15021: connect: connection refused" start-of-body=
Oct 30 19:55:21.683850 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:21.683474    5704 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c" podUID="af934c90-d9b7-427d-acce-67f9cc332a82" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.171:15021/healthz/ready\": dial tcp 10.134.0.171:15021: connect: connection refused"
Oct 30 19:55:22.795958 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:22.795915    5704 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:22.796676 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:22.796656    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:22.797691 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:22.797671    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:55:23.689109 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:23.689070    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:24.741185 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:24.741146    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf" podUID="f9b0ac2e-2d36-454f-8666-25acd2696603" containerName="payload-processing" containerID="cri-o://14e51e7912b90137b0167c88eff4ee26433a0ff350bcfb8a69704ccac3301948" gracePeriod=25
Oct 30 19:55:25.000122 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.000096    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"
Oct 30 19:55:25.042157 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.042132    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f9b0ac2e-2d36-454f-8666-25acd2696603-kube-api-access-9gb7m\" (UniqueName: \"kubernetes.io/projected/f9b0ac2e-2d36-454f-8666-25acd2696603-kube-api-access-9gb7m\") pod \"f9b0ac2e-2d36-454f-8666-25acd2696603\" (UID: \"f9b0ac2e-2d36-454f-8666-25acd2696603\") "
Oct 30 19:55:25.042286 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.042269    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f9b0ac2e-2d36-454f-8666-25acd2696603-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b0ac2e-2d36-454f-8666-25acd2696603-plugins-config-volume\") pod \"f9b0ac2e-2d36-454f-8666-25acd2696603\" (UID: \"f9b0ac2e-2d36-454f-8666-25acd2696603\") "
Oct 30 19:55:25.042605 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.042583    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9b0ac2e-2d36-454f-8666-25acd2696603-plugins-config-volume" pod "f9b0ac2e-2d36-454f-8666-25acd2696603" (UID: "f9b0ac2e-2d36-454f-8666-25acd2696603"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:55:25.044268 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.044248    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b0ac2e-2d36-454f-8666-25acd2696603-kube-api-access-9gb7m" pod "f9b0ac2e-2d36-454f-8666-25acd2696603" (UID: "f9b0ac2e-2d36-454f-8666-25acd2696603"). InnerVolumeSpecName "kube-api-access-9gb7m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:55:25.144489 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.144456    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gb7m\" (UniqueName: \"kubernetes.io/projected/f9b0ac2e-2d36-454f-8666-25acd2696603-kube-api-access-9gb7m\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:25.144489 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.144490    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b0ac2e-2d36-454f-8666-25acd2696603-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:55:25.855049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.855000    5704 generic.go:356] "Generic (PLEG): container finished" podID="f9b0ac2e-2d36-454f-8666-25acd2696603" containerID="14e51e7912b90137b0167c88eff4ee26433a0ff350bcfb8a69704ccac3301948" exitCode=0
Oct 30 19:55:25.855611 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.855071    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"
Oct 30 19:55:25.855611 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.855087    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf" event={"ID":"f9b0ac2e-2d36-454f-8666-25acd2696603","Type":"ContainerDied","Data":"14e51e7912b90137b0167c88eff4ee26433a0ff350bcfb8a69704ccac3301948"}
Oct 30 19:55:25.855611 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.855174    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf" event={"ID":"f9b0ac2e-2d36-454f-8666-25acd2696603","Type":"ContainerDied","Data":"5f0e5c3c75f061610b59862cb72055aef0c4646b237242957426b035bc5b326a"}
Oct 30 19:55:25.855611 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.855191    5704 scope.go:122] "RemoveContainer" containerID="14e51e7912b90137b0167c88eff4ee26433a0ff350bcfb8a69704ccac3301948"
Oct 30 19:55:25.864988 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.864968    5704 scope.go:122] "RemoveContainer" containerID="14e51e7912b90137b0167c88eff4ee26433a0ff350bcfb8a69704ccac3301948"
Oct 30 19:55:25.865324 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:55:25.865295    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14e51e7912b90137b0167c88eff4ee26433a0ff350bcfb8a69704ccac3301948\": container with ID starting with 14e51e7912b90137b0167c88eff4ee26433a0ff350bcfb8a69704ccac3301948 not found: ID does not exist" containerID="14e51e7912b90137b0167c88eff4ee26433a0ff350bcfb8a69704ccac3301948"
Oct 30 19:55:25.865426 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:25.865329    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14e51e7912b90137b0167c88eff4ee26433a0ff350bcfb8a69704ccac3301948"} err="failed to get container status \"14e51e7912b90137b0167c88eff4ee26433a0ff350bcfb8a69704ccac3301948\": rpc error: code = NotFound desc = could not find container \"14e51e7912b90137b0167c88eff4ee26433a0ff350bcfb8a69704ccac3301948\": container with ID starting with 14e51e7912b90137b0167c88eff4ee26433a0ff350bcfb8a69704ccac3301948 not found: ID does not exist"
Oct 30 19:55:26.011144 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:26.011116    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"]
Oct 30 19:55:26.056485 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:26.056461    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-4a36b7-b7c545c5b-5vdkf"]
Oct 30 19:55:27.179856 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:27.179823    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f9b0ac2e-2d36-454f-8666-25acd2696603" path="/var/lib/kubelet/pods/f9b0ac2e-2d36-454f-8666-25acd2696603/volumes"
Oct 30 19:55:28.787249 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:28.781559    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:30.819979 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:30.819946    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:31.003338 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:31.003301    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:33.383467 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:55:33.383432    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:55:33.781290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:33.781256    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:38.649220 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:38.649186    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:40.399171 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:40.399138    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:40.584689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:40.584654    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:43.263903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:43.263868    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:45.968001 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:45.967967    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:46.201624 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:46.201587    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:48.218522 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:48.218491    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:52.278944 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:52.278905    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:55:53.013253 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:53.013219    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:55:57.568437 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:55:57.568399    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:02.250907 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:02.250874    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:03.384296 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:56:03.384258    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:56:06.367042 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:06.367008    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:07.837596 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:07.837562    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:08.099283 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:08.099168    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:08.282523 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:08.282487    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:08.764885 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:08.764836    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:13.049967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:13.049928    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:14.111892 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:14.111851    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:15.563444 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:15.563400    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:15.748606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:15.748575    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:15.941953 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:15.941854    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:17.600077 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:17.600005    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:18.730060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:18.730013    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:22.470493 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:22.469841    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:23.931101 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:23.931070    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:56:27.371341 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:27.371289    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:29.837739 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:29.837704    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"]
Oct 30 19:56:30.050411 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:30.050380    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"]
Oct 30 19:56:30.264193 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:30.264159    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:30.477623 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:30.477584    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:31.992879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:31.992842    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:33.383410 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:56:33.383378    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:56:34.838725 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:34.838681    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb" podUID="0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9" containerName="payload-processing" containerID="cri-o://dd9d83fa494b7592f32753c26477bc8fb217ffc4f5f228067b0370e4ddf6020a" gracePeriod=25
Oct 30 19:56:35.050961 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:35.050917    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt" podUID="f67a70de-5e78-4a06-a16b-cd846189120d" containerName="payload-pre-processing" containerID="cri-o://fcb7f418d2c241253a44cf429ac8c39dc0100eae345be6128ab703904f807a9a" gracePeriod=25
Oct 30 19:56:35.731847 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:35.731814    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"]
Oct 30 19:56:35.732089 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:35.732043    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l" podUID="faa9d942-3670-49ef-b4bc-a3fe995a642f" containerName="maas-api" containerID="cri-o://ebe4b06d0c3acc28bd11a4c65faf557d7de85f527ed55ac80ec0e4a60df61d8f" gracePeriod=30
Oct 30 19:56:35.858945 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:35.858902    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:36.042408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.042381    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:56:36.125058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.121509    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-maas-api-tls\") pod \"faa9d942-3670-49ef-b4bc-a3fe995a642f\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") "
Oct 30 19:56:36.125058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.121781    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/faa9d942-3670-49ef-b4bc-a3fe995a642f-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa9d942-3670-49ef-b4bc-a3fe995a642f-openshift-service-ca\") pod \"faa9d942-3670-49ef-b4bc-a3fe995a642f\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") "
Oct 30 19:56:36.125058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.121868    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-metrics-certs\") pod \"faa9d942-3670-49ef-b4bc-a3fe995a642f\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") "
Oct 30 19:56:36.125058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.121907    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/faa9d942-3670-49ef-b4bc-a3fe995a642f-kube-api-access-mnjsq\" (UniqueName: \"kubernetes.io/projected/faa9d942-3670-49ef-b4bc-a3fe995a642f-kube-api-access-mnjsq\") pod \"faa9d942-3670-49ef-b4bc-a3fe995a642f\" (UID: \"faa9d942-3670-49ef-b4bc-a3fe995a642f\") "
Oct 30 19:56:36.131054 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.127253    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faa9d942-3670-49ef-b4bc-a3fe995a642f-openshift-service-ca" pod "faa9d942-3670-49ef-b4bc-a3fe995a642f" (UID: "faa9d942-3670-49ef-b4bc-a3fe995a642f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:36.131692 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.131634    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faa9d942-3670-49ef-b4bc-a3fe995a642f-kube-api-access-mnjsq" pod "faa9d942-3670-49ef-b4bc-a3fe995a642f" (UID: "faa9d942-3670-49ef-b4bc-a3fe995a642f"). InnerVolumeSpecName "kube-api-access-mnjsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:36.136635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.136603    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-metrics-certs" pod "faa9d942-3670-49ef-b4bc-a3fe995a642f" (UID: "faa9d942-3670-49ef-b4bc-a3fe995a642f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:56:36.143117 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.143088    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-maas-api-tls" pod "faa9d942-3670-49ef-b4bc-a3fe995a642f" (UID: "faa9d942-3670-49ef-b4bc-a3fe995a642f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:56:36.223290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.223256    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/faa9d942-3670-49ef-b4bc-a3fe995a642f-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:36.223290 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.223286    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:36.223488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.223300    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mnjsq\" (UniqueName: \"kubernetes.io/projected/faa9d942-3670-49ef-b4bc-a3fe995a642f-kube-api-access-mnjsq\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:36.223488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.223309    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/faa9d942-3670-49ef-b4bc-a3fe995a642f-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:36.358574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.358478    5704 generic.go:356] "Generic (PLEG): container finished" podID="faa9d942-3670-49ef-b4bc-a3fe995a642f" containerID="ebe4b06d0c3acc28bd11a4c65faf557d7de85f527ed55ac80ec0e4a60df61d8f" exitCode=0
Oct 30 19:56:36.358574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.358530    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"
Oct 30 19:56:36.358806 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.358567    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l" event={"ID":"faa9d942-3670-49ef-b4bc-a3fe995a642f","Type":"ContainerDied","Data":"ebe4b06d0c3acc28bd11a4c65faf557d7de85f527ed55ac80ec0e4a60df61d8f"}
Oct 30 19:56:36.358806 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.358624    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l" event={"ID":"faa9d942-3670-49ef-b4bc-a3fe995a642f","Type":"ContainerDied","Data":"dae5829ef140dc8d23d843559046fa897129490abc308e6fd06ad86eb4eb5bd3"}
Oct 30 19:56:36.358806 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.358647    5704 scope.go:122] "RemoveContainer" containerID="ebe4b06d0c3acc28bd11a4c65faf557d7de85f527ed55ac80ec0e4a60df61d8f"
Oct 30 19:56:36.369994 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.369975    5704 scope.go:122] "RemoveContainer" containerID="ebe4b06d0c3acc28bd11a4c65faf557d7de85f527ed55ac80ec0e4a60df61d8f"
Oct 30 19:56:36.370723 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:56:36.370701    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebe4b06d0c3acc28bd11a4c65faf557d7de85f527ed55ac80ec0e4a60df61d8f\": container with ID starting with ebe4b06d0c3acc28bd11a4c65faf557d7de85f527ed55ac80ec0e4a60df61d8f not found: ID does not exist" containerID="ebe4b06d0c3acc28bd11a4c65faf557d7de85f527ed55ac80ec0e4a60df61d8f"
Oct 30 19:56:36.370826 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.370732    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebe4b06d0c3acc28bd11a4c65faf557d7de85f527ed55ac80ec0e4a60df61d8f"} err="failed to get container status \"ebe4b06d0c3acc28bd11a4c65faf557d7de85f527ed55ac80ec0e4a60df61d8f\": rpc error: code = NotFound desc = could not find container \"ebe4b06d0c3acc28bd11a4c65faf557d7de85f527ed55ac80ec0e4a60df61d8f\": container with ID starting with ebe4b06d0c3acc28bd11a4c65faf557d7de85f527ed55ac80ec0e4a60df61d8f not found: ID does not exist"
Oct 30 19:56:36.512821 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.512794    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"]
Oct 30 19:56:36.517067 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.517022    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-b-4a36b7-6d6d897bd7-vc99l"]
Oct 30 19:56:36.971699 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.971665    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb" podUID="0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:56:36.971699 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:36.971696    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt" podUID="f67a70de-5e78-4a06-a16b-cd846189120d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:56:37.181422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:37.181390    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="faa9d942-3670-49ef-b4bc-a3fe995a642f" path="/var/lib/kubelet/pods/faa9d942-3670-49ef-b4bc-a3fe995a642f/volumes"
Oct 30 19:56:37.325455 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:37.325419    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:37.649963 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:37.649877    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:37.843422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:37.843385    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:38.388355 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:38.388317    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:42.584371 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:42.584292    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:44.817325 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:44.817280    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:45.049924 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:45.049893    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:45.266086 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:45.264606    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:46.699129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:46.699100    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:46.971210 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:46.971134    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb" podUID="0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:56:46.971357 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:46.971213    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt" podUID="f67a70de-5e78-4a06-a16b-cd846189120d" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:56:47.806057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:47.806004    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"]
Oct 30 19:56:47.806500 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:47.806201    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr" podUID="622d8410-59c1-4a09-acce-2aae15528bac" containerName="maas-api" containerID="cri-o://eaa36714e478ab47ed50e55b7fa34dce64db7a60fd35ef383fcc4c20c791cb82" gracePeriod=30
Oct 30 19:56:47.962883 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:47.962847    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:48.105144 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.105118    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:56:48.291126 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.291091    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/622d8410-59c1-4a09-acce-2aae15528bac-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/622d8410-59c1-4a09-acce-2aae15528bac-openshift-service-ca\") pod \"622d8410-59c1-4a09-acce-2aae15528bac\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") "
Oct 30 19:56:48.291320 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.291145    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-metrics-certs\" (UniqueName: \"kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-metrics-certs\") pod \"622d8410-59c1-4a09-acce-2aae15528bac\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") "
Oct 30 19:56:48.291320 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.291199    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/622d8410-59c1-4a09-acce-2aae15528bac-kube-api-access-6hhx8\" (UniqueName: \"kubernetes.io/projected/622d8410-59c1-4a09-acce-2aae15528bac-kube-api-access-6hhx8\") pod \"622d8410-59c1-4a09-acce-2aae15528bac\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") "
Oct 30 19:56:48.291442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.291335    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-maas-api-tls\") pod \"622d8410-59c1-4a09-acce-2aae15528bac\" (UID: \"622d8410-59c1-4a09-acce-2aae15528bac\") "
Oct 30 19:56:48.291532 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.291505    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/622d8410-59c1-4a09-acce-2aae15528bac-openshift-service-ca" pod "622d8410-59c1-4a09-acce-2aae15528bac" (UID: "622d8410-59c1-4a09-acce-2aae15528bac"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:48.292102 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.292049    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/622d8410-59c1-4a09-acce-2aae15528bac-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:48.293510 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.293477    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-metrics-certs" pod "622d8410-59c1-4a09-acce-2aae15528bac" (UID: "622d8410-59c1-4a09-acce-2aae15528bac"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:56:48.293956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.293919    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-maas-api-tls" pod "622d8410-59c1-4a09-acce-2aae15528bac" (UID: "622d8410-59c1-4a09-acce-2aae15528bac"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:56:48.293956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.293927    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/622d8410-59c1-4a09-acce-2aae15528bac-kube-api-access-6hhx8" pod "622d8410-59c1-4a09-acce-2aae15528bac" (UID: "622d8410-59c1-4a09-acce-2aae15528bac"). InnerVolumeSpecName "kube-api-access-6hhx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:48.314901 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.314873    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:48.393628 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.393543    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:48.393628 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.393578    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/622d8410-59c1-4a09-acce-2aae15528bac-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:48.393628 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.393588    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hhx8\" (UniqueName: \"kubernetes.io/projected/622d8410-59c1-4a09-acce-2aae15528bac-kube-api-access-6hhx8\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:48.434882 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.434845    5704 generic.go:356] "Generic (PLEG): container finished" podID="622d8410-59c1-4a09-acce-2aae15528bac" containerID="eaa36714e478ab47ed50e55b7fa34dce64db7a60fd35ef383fcc4c20c791cb82" exitCode=0
Oct 30 19:56:48.435072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.434895    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"
Oct 30 19:56:48.435072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.434922    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr" event={"ID":"622d8410-59c1-4a09-acce-2aae15528bac","Type":"ContainerDied","Data":"eaa36714e478ab47ed50e55b7fa34dce64db7a60fd35ef383fcc4c20c791cb82"}
Oct 30 19:56:48.435072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.434979    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr" event={"ID":"622d8410-59c1-4a09-acce-2aae15528bac","Type":"ContainerDied","Data":"4c7deb3423c45928d4cd44a12661f1315ff88208c97b8c76e5929307517f0d9f"}
Oct 30 19:56:48.435072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.435000    5704 scope.go:122] "RemoveContainer" containerID="eaa36714e478ab47ed50e55b7fa34dce64db7a60fd35ef383fcc4c20c791cb82"
Oct 30 19:56:48.449087 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.449022    5704 scope.go:122] "RemoveContainer" containerID="eaa36714e478ab47ed50e55b7fa34dce64db7a60fd35ef383fcc4c20c791cb82"
Oct 30 19:56:48.449392 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:56:48.449369    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eaa36714e478ab47ed50e55b7fa34dce64db7a60fd35ef383fcc4c20c791cb82\": container with ID starting with eaa36714e478ab47ed50e55b7fa34dce64db7a60fd35ef383fcc4c20c791cb82 not found: ID does not exist" containerID="eaa36714e478ab47ed50e55b7fa34dce64db7a60fd35ef383fcc4c20c791cb82"
Oct 30 19:56:48.449510 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.449395    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eaa36714e478ab47ed50e55b7fa34dce64db7a60fd35ef383fcc4c20c791cb82"} err="failed to get container status \"eaa36714e478ab47ed50e55b7fa34dce64db7a60fd35ef383fcc4c20c791cb82\": rpc error: code = NotFound desc = could not find container \"eaa36714e478ab47ed50e55b7fa34dce64db7a60fd35ef383fcc4c20c791cb82\": container with ID starting with eaa36714e478ab47ed50e55b7fa34dce64db7a60fd35ef383fcc4c20c791cb82 not found: ID does not exist"
Oct 30 19:56:48.581653 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.581602    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:48.590721 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.590696    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"]
Oct 30 19:56:48.606836 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:48.606780    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-7b6aee-79f4df76b7-htkhr"]
Oct 30 19:56:49.180315 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:49.180285    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="622d8410-59c1-4a09-acce-2aae15528bac" path="/var/lib/kubelet/pods/622d8410-59c1-4a09-acce-2aae15528bac/volumes"
Oct 30 19:56:49.220250 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:49.220226    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:49.592256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:49.592222    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:49.906110 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:49.904499    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:50.119335 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:50.119285    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:50.524428 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:50.524394    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:52.302073 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:52.302025    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:54.706665 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:54.706627    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:55.183602 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.183574    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"
Oct 30 19:56:55.289554 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.289465    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-plugins-config-volume\") pod \"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9\" (UID: \"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9\") "
Oct 30 19:56:55.289554 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.289506    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-kube-api-access-mgxxw\" (UniqueName: \"kubernetes.io/projected/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-kube-api-access-mgxxw\") pod \"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9\" (UID: \"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9\") "
Oct 30 19:56:55.289935 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.289908    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-plugins-config-volume" pod "0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9" (UID: "0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:55.290181 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.290160    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:55.291645 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.291619    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-kube-api-access-mgxxw" pod "0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9" (UID: "0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9"). InnerVolumeSpecName "kube-api-access-mgxxw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:55.308797 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.308774    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"
Oct 30 19:56:55.390831 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.390801    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f67a70de-5e78-4a06-a16b-cd846189120d-kube-api-access-nx9qv\" (UniqueName: \"kubernetes.io/projected/f67a70de-5e78-4a06-a16b-cd846189120d-kube-api-access-nx9qv\") pod \"f67a70de-5e78-4a06-a16b-cd846189120d\" (UID: \"f67a70de-5e78-4a06-a16b-cd846189120d\") "
Oct 30 19:56:55.390999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.390847    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f67a70de-5e78-4a06-a16b-cd846189120d-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f67a70de-5e78-4a06-a16b-cd846189120d-plugins-config-volume\") pod \"f67a70de-5e78-4a06-a16b-cd846189120d\" (UID: \"f67a70de-5e78-4a06-a16b-cd846189120d\") "
Oct 30 19:56:55.391329 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.391299    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f67a70de-5e78-4a06-a16b-cd846189120d-plugins-config-volume" pod "f67a70de-5e78-4a06-a16b-cd846189120d" (UID: "f67a70de-5e78-4a06-a16b-cd846189120d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:56:55.391533 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.391517    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgxxw\" (UniqueName: \"kubernetes.io/projected/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9-kube-api-access-mgxxw\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:55.391582 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.391536    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f67a70de-5e78-4a06-a16b-cd846189120d-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:55.392840 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.392811    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f67a70de-5e78-4a06-a16b-cd846189120d-kube-api-access-nx9qv" pod "f67a70de-5e78-4a06-a16b-cd846189120d" (UID: "f67a70de-5e78-4a06-a16b-cd846189120d"). InnerVolumeSpecName "kube-api-access-nx9qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:56:55.480490 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.480459    5704 generic.go:356] "Generic (PLEG): container finished" podID="f67a70de-5e78-4a06-a16b-cd846189120d" containerID="fcb7f418d2c241253a44cf429ac8c39dc0100eae345be6128ab703904f807a9a" exitCode=0
Oct 30 19:56:55.480690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.480502    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"
Oct 30 19:56:55.480690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.480536    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt" event={"ID":"f67a70de-5e78-4a06-a16b-cd846189120d","Type":"ContainerDied","Data":"fcb7f418d2c241253a44cf429ac8c39dc0100eae345be6128ab703904f807a9a"}
Oct 30 19:56:55.480690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.480601    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt" event={"ID":"f67a70de-5e78-4a06-a16b-cd846189120d","Type":"ContainerDied","Data":"82acc4a31de11a39d741f5ee4869e8a13b2ea42f514d11166117b49282d60538"}
Oct 30 19:56:55.480690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.480637    5704 scope.go:122] "RemoveContainer" containerID="fcb7f418d2c241253a44cf429ac8c39dc0100eae345be6128ab703904f807a9a"
Oct 30 19:56:55.482245 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.482220    5704 generic.go:356] "Generic (PLEG): container finished" podID="0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9" containerID="dd9d83fa494b7592f32753c26477bc8fb217ffc4f5f228067b0370e4ddf6020a" exitCode=0
Oct 30 19:56:55.482396 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.482264    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"
Oct 30 19:56:55.482396 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.482282    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb" event={"ID":"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9","Type":"ContainerDied","Data":"dd9d83fa494b7592f32753c26477bc8fb217ffc4f5f228067b0370e4ddf6020a"}
Oct 30 19:56:55.482396 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.482319    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb" event={"ID":"0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9","Type":"ContainerDied","Data":"aa31811aa3a11c1105101164721398d17ee61c5c790fb8472ca5f1a220b372d3"}
Oct 30 19:56:55.492343 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.492324    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx9qv\" (UniqueName: \"kubernetes.io/projected/f67a70de-5e78-4a06-a16b-cd846189120d-kube-api-access-nx9qv\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:56:55.492437 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.492418    5704 scope.go:122] "RemoveContainer" containerID="fcb7f418d2c241253a44cf429ac8c39dc0100eae345be6128ab703904f807a9a"
Oct 30 19:56:55.492710 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:56:55.492688    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcb7f418d2c241253a44cf429ac8c39dc0100eae345be6128ab703904f807a9a\": container with ID starting with fcb7f418d2c241253a44cf429ac8c39dc0100eae345be6128ab703904f807a9a not found: ID does not exist" containerID="fcb7f418d2c241253a44cf429ac8c39dc0100eae345be6128ab703904f807a9a"
Oct 30 19:56:55.492802 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.492721    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcb7f418d2c241253a44cf429ac8c39dc0100eae345be6128ab703904f807a9a"} err="failed to get container status \"fcb7f418d2c241253a44cf429ac8c39dc0100eae345be6128ab703904f807a9a\": rpc error: code = NotFound desc = could not find container \"fcb7f418d2c241253a44cf429ac8c39dc0100eae345be6128ab703904f807a9a\": container with ID starting with fcb7f418d2c241253a44cf429ac8c39dc0100eae345be6128ab703904f807a9a not found: ID does not exist"
Oct 30 19:56:55.492802 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.492747    5704 scope.go:122] "RemoveContainer" containerID="dd9d83fa494b7592f32753c26477bc8fb217ffc4f5f228067b0370e4ddf6020a"
Oct 30 19:56:55.502936 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.502917    5704 scope.go:122] "RemoveContainer" containerID="dd9d83fa494b7592f32753c26477bc8fb217ffc4f5f228067b0370e4ddf6020a"
Oct 30 19:56:55.503236 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:56:55.503214    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd9d83fa494b7592f32753c26477bc8fb217ffc4f5f228067b0370e4ddf6020a\": container with ID starting with dd9d83fa494b7592f32753c26477bc8fb217ffc4f5f228067b0370e4ddf6020a not found: ID does not exist" containerID="dd9d83fa494b7592f32753c26477bc8fb217ffc4f5f228067b0370e4ddf6020a"
Oct 30 19:56:55.503316 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.503246    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd9d83fa494b7592f32753c26477bc8fb217ffc4f5f228067b0370e4ddf6020a"} err="failed to get container status \"dd9d83fa494b7592f32753c26477bc8fb217ffc4f5f228067b0370e4ddf6020a\": rpc error: code = NotFound desc = could not find container \"dd9d83fa494b7592f32753c26477bc8fb217ffc4f5f228067b0370e4ddf6020a\": container with ID starting with dd9d83fa494b7592f32753c26477bc8fb217ffc4f5f228067b0370e4ddf6020a not found: ID does not exist"
Oct 30 19:56:55.567538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.567473    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:55.635767 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.635732    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"]
Oct 30 19:56:55.639186 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.639163    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-fe7634-7688777f9b-9rwnb"]
Oct 30 19:56:55.745493 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.745464    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"]
Oct 30 19:56:55.750855 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.750827    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-fe7634-dbd867fbc-7s7jt"]
Oct 30 19:56:55.780777 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.780736    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:55.994571 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:55.994541    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:57.179927 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:57.179893    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9" path="/var/lib/kubelet/pods/0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9/volumes"
Oct 30 19:56:57.180348 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:57.180323    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f67a70de-5e78-4a06-a16b-cd846189120d" path="/var/lib/kubelet/pods/f67a70de-5e78-4a06-a16b-cd846189120d/volumes"
Oct 30 19:56:57.348579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:57.348542    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:57.569011 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:57.568976    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:57.675675 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:57.675640    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"]
Oct 30 19:56:57.675867 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:57.675848    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c" podUID="af934c90-d9b7-427d-acce-67f9cc332a82" containerName="istio-proxy" containerID="cri-o://e8dff82dec3d9d865de729409d763cc5d68add06ce3ce83d094de13bcc93b43b" gracePeriod=30
Oct 30 19:56:57.701749 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:57.701724    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"]
Oct 30 19:56:57.701927 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:57.701905    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn" podUID="fa6c415c-9fee-4b1e-979f-5764d5cba9a4" containerName="istio-proxy" containerID="cri-o://f7ebb944bd6dc11e183dd72e7cfc2c178bd88aef147906596d800603ed08b442" gracePeriod=30
Oct 30 19:56:57.811062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:57.810420    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:58.789524 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:58.789490    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"]
Oct 30 19:56:59.161204 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:59.161113    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:59.355704 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:59.355668    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:56:59.665243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:56:59.665209    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:01.707761 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:01.707730    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:57:03.030793 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.029375    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:57:03.051245 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.051222    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:57:03.130666 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.130634    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-istio-token\" (UniqueName: \"kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-istio-token\") pod \"af934c90-d9b7-427d-acce-67f9cc332a82\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") "
Oct 30 19:57:03.130834 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.130687    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-data\") pod \"af934c90-d9b7-427d-acce-67f9cc332a82\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") "
Oct 30 19:57:03.130897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.130876    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-credential-socket\") pod \"af934c90-d9b7-427d-acce-67f9cc332a82\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") "
Oct 30 19:57:03.130964 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.130933    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-data" pod "af934c90-d9b7-427d-acce-67f9cc332a82" (UID: "af934c90-d9b7-427d-acce-67f9cc332a82"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.130964 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.130955    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-envoy\") pod \"af934c90-d9b7-427d-acce-67f9cc332a82\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") "
Oct 30 19:57:03.131121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.131023    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-kube-api-access-sqzt2\" (UniqueName: \"kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-kube-api-access-sqzt2\") pod \"af934c90-d9b7-427d-acce-67f9cc332a82\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") "
Oct 30 19:57:03.131121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.131104    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/af934c90-d9b7-427d-acce-67f9cc332a82-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af934c90-d9b7-427d-acce-67f9cc332a82-istiod-ca-cert\") pod \"af934c90-d9b7-427d-acce-67f9cc332a82\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") "
Oct 30 19:57:03.131234 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.131158    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-credential-socket" pod "af934c90-d9b7-427d-acce-67f9cc332a82" (UID: "af934c90-d9b7-427d-acce-67f9cc332a82"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.131234 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.131170    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/af934c90-d9b7-427d-acce-67f9cc332a82-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af934c90-d9b7-427d-acce-67f9cc332a82-istio-podinfo\") pod \"af934c90-d9b7-427d-acce-67f9cc332a82\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") "
Oct 30 19:57:03.131354 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.131314    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-certs\") pod \"af934c90-d9b7-427d-acce-67f9cc332a82\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") "
Oct 30 19:57:03.131414 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.131387    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-socket\") pod \"af934c90-d9b7-427d-acce-67f9cc332a82\" (UID: \"af934c90-d9b7-427d-acce-67f9cc332a82\") "
Oct 30 19:57:03.131622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.131592    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-certs" pod "af934c90-d9b7-427d-acce-67f9cc332a82" (UID: "af934c90-d9b7-427d-acce-67f9cc332a82"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.131622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.131608    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-socket" pod "af934c90-d9b7-427d-acce-67f9cc332a82" (UID: "af934c90-d9b7-427d-acce-67f9cc332a82"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.131820 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.131796    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af934c90-d9b7-427d-acce-67f9cc332a82-istiod-ca-cert" pod "af934c90-d9b7-427d-acce-67f9cc332a82" (UID: "af934c90-d9b7-427d-acce-67f9cc332a82"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:57:03.132056 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.132009    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.132144 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.132059    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.132144 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.132075    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af934c90-d9b7-427d-acce-67f9cc332a82-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.132144 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.132089    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.132144 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.132103    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.133124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.133095    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-istio-token" pod "af934c90-d9b7-427d-acce-67f9cc332a82" (UID: "af934c90-d9b7-427d-acce-67f9cc332a82"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:03.133124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.133114    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-envoy" pod "af934c90-d9b7-427d-acce-67f9cc332a82" (UID: "af934c90-d9b7-427d-acce-67f9cc332a82"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.133266 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.133181    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-kube-api-access-sqzt2" pod "af934c90-d9b7-427d-acce-67f9cc332a82" (UID: "af934c90-d9b7-427d-acce-67f9cc332a82"). InnerVolumeSpecName "kube-api-access-sqzt2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:03.133307 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.133277    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/af934c90-d9b7-427d-acce-67f9cc332a82-istio-podinfo" pod "af934c90-d9b7-427d-acce-67f9cc332a82" (UID: "af934c90-d9b7-427d-acce-67f9cc332a82"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:57:03.233190 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.233116    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-socket\") pod \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") "
Oct 30 19:57:03.233190 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.233162    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-data\") pod \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") "
Oct 30 19:57:03.233421 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.233199    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-kube-api-access-ppgkf\" (UniqueName: \"kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-kube-api-access-ppgkf\") pod \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") "
Oct 30 19:57:03.233421 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.233226    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-certs\") pod \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") "
Oct 30 19:57:03.233421 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.233245    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-envoy\") pod \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") "
Oct 30 19:57:03.233421 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.233298    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-podinfo\") pod \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") "
Oct 30 19:57:03.233656 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.233423    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-socket" pod "fa6c415c-9fee-4b1e-979f-5764d5cba9a4" (UID: "fa6c415c-9fee-4b1e-979f-5764d5cba9a4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.233656 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.233435    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-token\" (UniqueName: \"kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-token\") pod \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") "
Oct 30 19:57:03.233656 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.233550    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-credential-socket\") pod \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") "
Oct 30 19:57:03.233656 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.233637    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istiod-ca-cert\") pod \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\" (UID: \"fa6c415c-9fee-4b1e-979f-5764d5cba9a4\") "
Oct 30 19:57:03.233872 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.233799    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-certs" pod "fa6c415c-9fee-4b1e-979f-5764d5cba9a4" (UID: "fa6c415c-9fee-4b1e-979f-5764d5cba9a4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.234285 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.234235    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqzt2\" (UniqueName: \"kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-kube-api-access-sqzt2\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.234285 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.234260    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af934c90-d9b7-427d-acce-67f9cc332a82-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.234285 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.234279    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.234910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.234295    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.234910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.234310    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af934c90-d9b7-427d-acce-67f9cc332a82-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.234910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.234312    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-data" pod "fa6c415c-9fee-4b1e-979f-5764d5cba9a4" (UID: "fa6c415c-9fee-4b1e-979f-5764d5cba9a4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.234910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.234325    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af934c90-d9b7-427d-acce-67f9cc332a82-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.234910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.234633    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istiod-ca-cert" pod "fa6c415c-9fee-4b1e-979f-5764d5cba9a4" (UID: "fa6c415c-9fee-4b1e-979f-5764d5cba9a4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:57:03.234910 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.234749    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-credential-socket" pod "fa6c415c-9fee-4b1e-979f-5764d5cba9a4" (UID: "fa6c415c-9fee-4b1e-979f-5764d5cba9a4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.236143 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.236118    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-kube-api-access-ppgkf" pod "fa6c415c-9fee-4b1e-979f-5764d5cba9a4" (UID: "fa6c415c-9fee-4b1e-979f-5764d5cba9a4"). InnerVolumeSpecName "kube-api-access-ppgkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:03.236229 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.236154    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-token" pod "fa6c415c-9fee-4b1e-979f-5764d5cba9a4" (UID: "fa6c415c-9fee-4b1e-979f-5764d5cba9a4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:03.236317 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.236290    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-envoy" pod "fa6c415c-9fee-4b1e-979f-5764d5cba9a4" (UID: "fa6c415c-9fee-4b1e-979f-5764d5cba9a4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:03.237179 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.237149    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-podinfo" pod "fa6c415c-9fee-4b1e-979f-5764d5cba9a4" (UID: "fa6c415c-9fee-4b1e-979f-5764d5cba9a4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:57:03.335222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.335180    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.335222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.335212    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.335222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.335222    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.335222 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.335231    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.335579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.335240    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppgkf\" (UniqueName: \"kubernetes.io/projected/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-kube-api-access-ppgkf\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.335579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.335249    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.335579 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.335261    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fa6c415c-9fee-4b1e-979f-5764d5cba9a4-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:03.383577 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:57:03.383540    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:57:03.534232 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.534191    5704 generic.go:356] "Generic (PLEG): container finished" podID="af934c90-d9b7-427d-acce-67f9cc332a82" containerID="e8dff82dec3d9d865de729409d763cc5d68add06ce3ce83d094de13bcc93b43b" exitCode=0
Oct 30 19:57:03.536219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.534244    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"
Oct 30 19:57:03.536219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.534276    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c" event={"ID":"af934c90-d9b7-427d-acce-67f9cc332a82","Type":"ContainerDied","Data":"e8dff82dec3d9d865de729409d763cc5d68add06ce3ce83d094de13bcc93b43b"}
Oct 30 19:57:03.536219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.534336    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c" event={"ID":"af934c90-d9b7-427d-acce-67f9cc332a82","Type":"ContainerDied","Data":"b74ab380f4f9a5b918a71311b81068066b8588fbeb8710702ab5c5912efcdbbf"}
Oct 30 19:57:03.536219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.534365    5704 scope.go:122] "RemoveContainer" containerID="e8dff82dec3d9d865de729409d763cc5d68add06ce3ce83d094de13bcc93b43b"
Oct 30 19:57:03.536219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.535678    5704 generic.go:356] "Generic (PLEG): container finished" podID="fa6c415c-9fee-4b1e-979f-5764d5cba9a4" containerID="f7ebb944bd6dc11e183dd72e7cfc2c178bd88aef147906596d800603ed08b442" exitCode=0
Oct 30 19:57:03.536219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.535710    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"
Oct 30 19:57:03.536219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.535756    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn" event={"ID":"fa6c415c-9fee-4b1e-979f-5764d5cba9a4","Type":"ContainerDied","Data":"f7ebb944bd6dc11e183dd72e7cfc2c178bd88aef147906596d800603ed08b442"}
Oct 30 19:57:03.536219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.535806    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn" event={"ID":"fa6c415c-9fee-4b1e-979f-5764d5cba9a4","Type":"ContainerDied","Data":"11c548d641b71fbad8cc8f92a5889b45b24611f584f2bde7400a904e0a71d167"}
Oct 30 19:57:03.546210 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.546194    5704 scope.go:122] "RemoveContainer" containerID="e8dff82dec3d9d865de729409d763cc5d68add06ce3ce83d094de13bcc93b43b"
Oct 30 19:57:03.546457 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:57:03.546441    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8dff82dec3d9d865de729409d763cc5d68add06ce3ce83d094de13bcc93b43b\": container with ID starting with e8dff82dec3d9d865de729409d763cc5d68add06ce3ce83d094de13bcc93b43b not found: ID does not exist" containerID="e8dff82dec3d9d865de729409d763cc5d68add06ce3ce83d094de13bcc93b43b"
Oct 30 19:57:03.546513 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.546464    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8dff82dec3d9d865de729409d763cc5d68add06ce3ce83d094de13bcc93b43b"} err="failed to get container status \"e8dff82dec3d9d865de729409d763cc5d68add06ce3ce83d094de13bcc93b43b\": rpc error: code = NotFound desc = could not find container \"e8dff82dec3d9d865de729409d763cc5d68add06ce3ce83d094de13bcc93b43b\": container with ID starting with e8dff82dec3d9d865de729409d763cc5d68add06ce3ce83d094de13bcc93b43b not found: ID does not exist"
Oct 30 19:57:03.546513 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.546479    5704 scope.go:122] "RemoveContainer" containerID="f7ebb944bd6dc11e183dd72e7cfc2c178bd88aef147906596d800603ed08b442"
Oct 30 19:57:03.555156 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.555140    5704 scope.go:122] "RemoveContainer" containerID="f7ebb944bd6dc11e183dd72e7cfc2c178bd88aef147906596d800603ed08b442"
Oct 30 19:57:03.555414 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:57:03.555391    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7ebb944bd6dc11e183dd72e7cfc2c178bd88aef147906596d800603ed08b442\": container with ID starting with f7ebb944bd6dc11e183dd72e7cfc2c178bd88aef147906596d800603ed08b442 not found: ID does not exist" containerID="f7ebb944bd6dc11e183dd72e7cfc2c178bd88aef147906596d800603ed08b442"
Oct 30 19:57:03.555467 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.555421    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7ebb944bd6dc11e183dd72e7cfc2c178bd88aef147906596d800603ed08b442"} err="failed to get container status \"f7ebb944bd6dc11e183dd72e7cfc2c178bd88aef147906596d800603ed08b442\": rpc error: code = NotFound desc = could not find container \"f7ebb944bd6dc11e183dd72e7cfc2c178bd88aef147906596d800603ed08b442\": container with ID starting with f7ebb944bd6dc11e183dd72e7cfc2c178bd88aef147906596d800603ed08b442 not found: ID does not exist"
Oct 30 19:57:03.692768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.692740    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"]
Oct 30 19:57:03.738874 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.738842    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-qr56c"]
Oct 30 19:57:03.790622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.790548    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7" podUID="6fdbaa8d-ad3f-4202-9ddc-e208538a14ac" containerName="payload-processing" containerID="cri-o://3658ce1f7bddfff7b91b2347090369c6e81254b1a96d389bafe64d4a761631ad" gracePeriod=25
Oct 30 19:57:03.810947 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.810918    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"]
Oct 30 19:57:03.814720 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:03.814697    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-a-7b6aee-openshift-default-6fddfcd74f-xs2wn"]
Oct 30 19:57:04.405501 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:04.405468    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7" podUID="6fdbaa8d-ad3f-4202-9ddc-e208538a14ac" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:57:04.682212 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:04.682135    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:05.180361 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:05.180331    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="af934c90-d9b7-427d-acce-67f9cc332a82" path="/var/lib/kubelet/pods/af934c90-d9b7-427d-acce-67f9cc332a82/volumes"
Oct 30 19:57:05.180775 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:05.180761    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa6c415c-9fee-4b1e-979f-5764d5cba9a4" path="/var/lib/kubelet/pods/fa6c415c-9fee-4b1e-979f-5764d5cba9a4/volumes"
Oct 30 19:57:08.526241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:08.524635    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:08.756154 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:08.756125    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:09.374087 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:09.374023    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:10.420000 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:10.419967    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"]
Oct 30 19:57:10.803827 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:10.803798    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:10.988163 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:10.988127    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:13.976498 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:13.976461    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:14.405126 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:14.405092    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7" podUID="6fdbaa8d-ad3f-4202-9ddc-e208538a14ac" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:57:15.420681 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:15.420613    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59" podUID="1daee16d-a4f6-4f87-b792-b3866450b2b2" containerName="payload-processing" containerID="cri-o://aa085dacb9181d3bef969accfa723226c66de6cb12bed736bab9996b47bc3cc2" gracePeriod=25
Oct 30 19:57:15.610213 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:15.610185    5704 generic.go:356] "Generic (PLEG): container finished" podID="1daee16d-a4f6-4f87-b792-b3866450b2b2" containerID="aa085dacb9181d3bef969accfa723226c66de6cb12bed736bab9996b47bc3cc2" exitCode=0
Oct 30 19:57:15.610352 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:15.610255    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59" event={"ID":"1daee16d-a4f6-4f87-b792-b3866450b2b2","Type":"ContainerDied","Data":"aa085dacb9181d3bef969accfa723226c66de6cb12bed736bab9996b47bc3cc2"}
Oct 30 19:57:15.693444 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:15.693405    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"
Oct 30 19:57:15.773911 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:15.773882    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1daee16d-a4f6-4f87-b792-b3866450b2b2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1daee16d-a4f6-4f87-b792-b3866450b2b2-plugins-config-volume\") pod \"1daee16d-a4f6-4f87-b792-b3866450b2b2\" (UID: \"1daee16d-a4f6-4f87-b792-b3866450b2b2\") "
Oct 30 19:57:15.774085 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:15.773924    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1daee16d-a4f6-4f87-b792-b3866450b2b2-kube-api-access-srt2b\" (UniqueName: \"kubernetes.io/projected/1daee16d-a4f6-4f87-b792-b3866450b2b2-kube-api-access-srt2b\") pod \"1daee16d-a4f6-4f87-b792-b3866450b2b2\" (UID: \"1daee16d-a4f6-4f87-b792-b3866450b2b2\") "
Oct 30 19:57:15.774306 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:15.774281    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1daee16d-a4f6-4f87-b792-b3866450b2b2-plugins-config-volume" pod "1daee16d-a4f6-4f87-b792-b3866450b2b2" (UID: "1daee16d-a4f6-4f87-b792-b3866450b2b2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:57:15.774464 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:15.774447    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1daee16d-a4f6-4f87-b792-b3866450b2b2-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:15.775934 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:15.775915    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1daee16d-a4f6-4f87-b792-b3866450b2b2-kube-api-access-srt2b" pod "1daee16d-a4f6-4f87-b792-b3866450b2b2" (UID: "1daee16d-a4f6-4f87-b792-b3866450b2b2"). InnerVolumeSpecName "kube-api-access-srt2b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:15.875822 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:15.875793    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srt2b\" (UniqueName: \"kubernetes.io/projected/1daee16d-a4f6-4f87-b792-b3866450b2b2-kube-api-access-srt2b\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:16.621234 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:16.621199    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"
Oct 30 19:57:16.621710 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:16.621206    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59" event={"ID":"1daee16d-a4f6-4f87-b792-b3866450b2b2","Type":"ContainerDied","Data":"568c235d3eb781d0bcb5883e2080161b0bd790fe48a24b5f6477cf02dda6a82a"}
Oct 30 19:57:16.621710 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:16.621392    5704 scope.go:122] "RemoveContainer" containerID="aa085dacb9181d3bef969accfa723226c66de6cb12bed736bab9996b47bc3cc2"
Oct 30 19:57:16.781637 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:16.781553    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"]
Oct 30 19:57:16.822598 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:16.822563    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w0-2186d8-58d5b469c4-zcx59"]
Oct 30 19:57:17.180472 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:17.180439    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1daee16d-a4f6-4f87-b792-b3866450b2b2" path="/var/lib/kubelet/pods/1daee16d-a4f6-4f87-b792-b3866450b2b2/volumes"
Oct 30 19:57:17.949094 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:17.949045    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:18.142960 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:18.142923    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:19.464360 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:19.464326    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:19.701131 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:19.701097    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:19.872503 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:19.872469    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:20.416495 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:20.416462    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:20.719653 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:20.719555    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:24.049076 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.048973    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"
Oct 30 19:57:24.173635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.173611    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-plugins-config-volume\") pod \"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac\" (UID: \"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac\") "
Oct 30 19:57:24.173801 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.173652    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-kube-api-access-rz5v5\" (UniqueName: \"kubernetes.io/projected/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-kube-api-access-rz5v5\") pod \"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac\" (UID: \"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac\") "
Oct 30 19:57:24.174000 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.173977    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-plugins-config-volume" pod "6fdbaa8d-ad3f-4202-9ddc-e208538a14ac" (UID: "6fdbaa8d-ad3f-4202-9ddc-e208538a14ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:57:24.174233 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.174214    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:24.175780 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.175757    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-kube-api-access-rz5v5" pod "6fdbaa8d-ad3f-4202-9ddc-e208538a14ac" (UID: "6fdbaa8d-ad3f-4202-9ddc-e208538a14ac"). InnerVolumeSpecName "kube-api-access-rz5v5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:24.275413 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.275383    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rz5v5\" (UniqueName: \"kubernetes.io/projected/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac-kube-api-access-rz5v5\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:24.660615 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.660582    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:24.693062 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.692986    5704 generic.go:356] "Generic (PLEG): container finished" podID="6fdbaa8d-ad3f-4202-9ddc-e208538a14ac" containerID="3658ce1f7bddfff7b91b2347090369c6e81254b1a96d389bafe64d4a761631ad" exitCode=0
Oct 30 19:57:24.693270 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.693237    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7" event={"ID":"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac","Type":"ContainerDied","Data":"3658ce1f7bddfff7b91b2347090369c6e81254b1a96d389bafe64d4a761631ad"}
Oct 30 19:57:24.693339 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.693324    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7" event={"ID":"6fdbaa8d-ad3f-4202-9ddc-e208538a14ac","Type":"ContainerDied","Data":"d937a59edfed4cb3ebbf81ac178e871cf8b08ca088fbd6fa6cf523f2b45d6a36"}
Oct 30 19:57:24.693393 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.693363    5704 scope.go:122] "RemoveContainer" containerID="3658ce1f7bddfff7b91b2347090369c6e81254b1a96d389bafe64d4a761631ad"
Oct 30 19:57:24.693587 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.693567    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"
Oct 30 19:57:24.708462 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.708444    5704 scope.go:122] "RemoveContainer" containerID="3658ce1f7bddfff7b91b2347090369c6e81254b1a96d389bafe64d4a761631ad"
Oct 30 19:57:24.708749 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:57:24.708731    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3658ce1f7bddfff7b91b2347090369c6e81254b1a96d389bafe64d4a761631ad\": container with ID starting with 3658ce1f7bddfff7b91b2347090369c6e81254b1a96d389bafe64d4a761631ad not found: ID does not exist" containerID="3658ce1f7bddfff7b91b2347090369c6e81254b1a96d389bafe64d4a761631ad"
Oct 30 19:57:24.708793 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.708758    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3658ce1f7bddfff7b91b2347090369c6e81254b1a96d389bafe64d4a761631ad"} err="failed to get container status \"3658ce1f7bddfff7b91b2347090369c6e81254b1a96d389bafe64d4a761631ad\": rpc error: code = NotFound desc = could not find container \"3658ce1f7bddfff7b91b2347090369c6e81254b1a96d389bafe64d4a761631ad\": container with ID starting with 3658ce1f7bddfff7b91b2347090369c6e81254b1a96d389bafe64d4a761631ad not found: ID does not exist"
Oct 30 19:57:24.847615 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.847586    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"]
Oct 30 19:57:24.852374 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:24.852351    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-db06cf-7dd7494956-zjsx7"]
Oct 30 19:57:25.179507 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:25.179476    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6fdbaa8d-ad3f-4202-9ddc-e208538a14ac" path="/var/lib/kubelet/pods/6fdbaa8d-ad3f-4202-9ddc-e208538a14ac/volumes"
Oct 30 19:57:27.487889 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:27.487856    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:27.682441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:27.682410    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:27.882633 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:27.882597    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:29.563559 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:29.560850    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:33.383581 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:57:33.383545    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:57:33.459253 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:33.459220    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:57:34.021576 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:34.021532    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:35.554724 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:35.554690    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:35.780898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:35.780863    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:35.971650 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:35.971567    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:36.500955 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:36.500921    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:40.121346 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.121311    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"]
Oct 30 19:57:40.121755 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.121499    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6" podUID="2f63052b-ba80-4c6e-9a7c-4413eebb990b" containerName="maas-api" containerID="cri-o://ca77e355714ec0cd5a4744f9b09c72e5a72cb9dab6a5c3ebd972c437cc594ff8" gracePeriod=30
Oct 30 19:57:40.265567 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.265534    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:40.396273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.396248    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:57:40.494589 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.494560    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2f63052b-ba80-4c6e-9a7c-4413eebb990b-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f63052b-ba80-4c6e-9a7c-4413eebb990b-openshift-service-ca\") pod \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") "
Oct 30 19:57:40.494760 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.494655    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-maas-api-tls\") pod \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") "
Oct 30 19:57:40.494809 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.494759    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2f63052b-ba80-4c6e-9a7c-4413eebb990b-kube-api-access-psbr4\" (UniqueName: \"kubernetes.io/projected/2f63052b-ba80-4c6e-9a7c-4413eebb990b-kube-api-access-psbr4\") pod \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") "
Oct 30 19:57:40.494809 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.494779    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-metrics-certs\") pod \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\" (UID: \"2f63052b-ba80-4c6e-9a7c-4413eebb990b\") "
Oct 30 19:57:40.494922 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.494880    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f63052b-ba80-4c6e-9a7c-4413eebb990b-openshift-service-ca" pod "2f63052b-ba80-4c6e-9a7c-4413eebb990b" (UID: "2f63052b-ba80-4c6e-9a7c-4413eebb990b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:57:40.495164 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.495142    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f63052b-ba80-4c6e-9a7c-4413eebb990b-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:40.496865 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.496824    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f63052b-ba80-4c6e-9a7c-4413eebb990b-kube-api-access-psbr4" pod "2f63052b-ba80-4c6e-9a7c-4413eebb990b" (UID: "2f63052b-ba80-4c6e-9a7c-4413eebb990b"). InnerVolumeSpecName "kube-api-access-psbr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:40.496865 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.496834    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-maas-api-tls" pod "2f63052b-ba80-4c6e-9a7c-4413eebb990b" (UID: "2f63052b-ba80-4c6e-9a7c-4413eebb990b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:57:40.497013 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.496857    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-metrics-certs" pod "2f63052b-ba80-4c6e-9a7c-4413eebb990b" (UID: "2f63052b-ba80-4c6e-9a7c-4413eebb990b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:57:40.596560 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.596527    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:40.596560 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.596554    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psbr4\" (UniqueName: \"kubernetes.io/projected/2f63052b-ba80-4c6e-9a7c-4413eebb990b-kube-api-access-psbr4\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:40.596758 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.596566    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2f63052b-ba80-4c6e-9a7c-4413eebb990b-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:40.728878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.728797    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:40.789093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.789022    5704 generic.go:356] "Generic (PLEG): container finished" podID="2f63052b-ba80-4c6e-9a7c-4413eebb990b" containerID="ca77e355714ec0cd5a4744f9b09c72e5a72cb9dab6a5c3ebd972c437cc594ff8" exitCode=0
Oct 30 19:57:40.789093 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.789092    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"
Oct 30 19:57:40.789350 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.789114    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6" event={"ID":"2f63052b-ba80-4c6e-9a7c-4413eebb990b","Type":"ContainerDied","Data":"ca77e355714ec0cd5a4744f9b09c72e5a72cb9dab6a5c3ebd972c437cc594ff8"}
Oct 30 19:57:40.789350 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.789169    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6" event={"ID":"2f63052b-ba80-4c6e-9a7c-4413eebb990b","Type":"ContainerDied","Data":"07ab22bc8419c017c65d62a4dc914f060ad275734aa874b516405921da11474a"}
Oct 30 19:57:40.789350 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.789196    5704 scope.go:122] "RemoveContainer" containerID="ca77e355714ec0cd5a4744f9b09c72e5a72cb9dab6a5c3ebd972c437cc594ff8"
Oct 30 19:57:40.799135 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.799116    5704 scope.go:122] "RemoveContainer" containerID="ca77e355714ec0cd5a4744f9b09c72e5a72cb9dab6a5c3ebd972c437cc594ff8"
Oct 30 19:57:40.799421 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:57:40.799403    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca77e355714ec0cd5a4744f9b09c72e5a72cb9dab6a5c3ebd972c437cc594ff8\": container with ID starting with ca77e355714ec0cd5a4744f9b09c72e5a72cb9dab6a5c3ebd972c437cc594ff8 not found: ID does not exist" containerID="ca77e355714ec0cd5a4744f9b09c72e5a72cb9dab6a5c3ebd972c437cc594ff8"
Oct 30 19:57:40.799479 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.799428    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca77e355714ec0cd5a4744f9b09c72e5a72cb9dab6a5c3ebd972c437cc594ff8"} err="failed to get container status \"ca77e355714ec0cd5a4744f9b09c72e5a72cb9dab6a5c3ebd972c437cc594ff8\": rpc error: code = NotFound desc = could not find container \"ca77e355714ec0cd5a4744f9b09c72e5a72cb9dab6a5c3ebd972c437cc594ff8\": container with ID starting with ca77e355714ec0cd5a4744f9b09c72e5a72cb9dab6a5c3ebd972c437cc594ff8 not found: ID does not exist"
Oct 30 19:57:40.909262 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.909229    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:40.984472 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.984392    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"]
Oct 30 19:57:40.998170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:40.998144    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-fe7634-747dc87f8f-xnsh6"]
Oct 30 19:57:41.181009 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:41.180975    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2f63052b-ba80-4c6e-9a7c-4413eebb990b" path="/var/lib/kubelet/pods/2f63052b-ba80-4c6e-9a7c-4413eebb990b/volumes"
Oct 30 19:57:41.859055 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:41.857835    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:42.062563 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:42.062529    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:42.249501 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:42.249467    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:42.767257 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:42.767225    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:43.300947 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:43.300906    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:43.504542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:43.504510    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:43.623739 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:43.623659    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"]
Oct 30 19:57:43.623890 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:43.623846    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerName="istio-proxy" containerID="cri-o://2fe03da0147966fb1e6b653c0505b5ffa0d0d0945e691ea376ba312f0aaf31a5" gracePeriod=30
Oct 30 19:57:43.710007 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:43.709966    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:47.215342 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:47.215299    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:48.842956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:48.842919    5704 generic.go:356] "Generic (PLEG): container finished" podID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerID="2fe03da0147966fb1e6b653c0505b5ffa0d0d0945e691ea376ba312f0aaf31a5" exitCode=0
Oct 30 19:57:48.843417 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:48.842984    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" event={"ID":"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce","Type":"ContainerDied","Data":"2fe03da0147966fb1e6b653c0505b5ffa0d0d0945e691ea376ba312f0aaf31a5"}
Oct 30 19:57:48.843417 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:48.843079    5704 scope.go:122] "RemoveContainer" containerID="62a43352f0135bbcbc52dbabfea0482e953e6ae252895e875f2c94edeed4b790"
Oct 30 19:57:48.896000 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:48.895948    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:57:49.008476 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.008448    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-kube-api-access-j9wkt\" (UniqueName: \"kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-kube-api-access-j9wkt\") pod \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") "
Oct 30 19:57:49.008635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.008509    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-socket\") pod \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") "
Oct 30 19:57:49.008635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.008539    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-podinfo\") pod \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") "
Oct 30 19:57:49.008635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.008562    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-token\" (UniqueName: \"kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-token\") pod \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") "
Oct 30 19:57:49.008635 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.008591    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-certs\") pod \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") "
Oct 30 19:57:49.008897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.008658    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istiod-ca-cert\") pod \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") "
Oct 30 19:57:49.008897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.008723    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-data\") pod \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") "
Oct 30 19:57:49.008897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.008732    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-socket" pod "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" (UID: "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:49.008897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.008755    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-credential-socket\") pod \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") "
Oct 30 19:57:49.008897 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.008794    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-envoy\") pod \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\" (UID: \"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce\") "
Oct 30 19:57:49.009195 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.009063    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-data" pod "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" (UID: "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:49.009195 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.009113    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-credential-socket" pod "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" (UID: "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:49.009195 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.009129    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-certs" pod "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" (UID: "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:49.009358 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.009323    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:49.009358 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.009343    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:49.009358 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.009341    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istiod-ca-cert" pod "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" (UID: "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:57:49.009358 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.009354    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:49.009580 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.009363    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:49.010861 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.010839    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-kube-api-access-j9wkt" pod "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" (UID: "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce"). InnerVolumeSpecName "kube-api-access-j9wkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:49.010861 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.010852    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-podinfo" pod "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" (UID: "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:57:49.011071 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.011055    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-envoy" pod "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" (UID: "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:57:49.011344 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.011321    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-token" pod "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" (UID: "8f538c70-f8bf-4a0a-bcc0-6800a8b495ce"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:57:49.110992 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.110957    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9wkt\" (UniqueName: \"kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-kube-api-access-j9wkt\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:49.110992 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.110994    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:49.111259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.111008    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:49.111259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.111023    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:49.111259 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.111053    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:57:49.850008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.849972    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp" event={"ID":"8f538c70-f8bf-4a0a-bcc0-6800a8b495ce","Type":"ContainerDied","Data":"adb247c73ee57a80b5850ebcb620bb2f6c7b8ca77c0a851505ebf20191ebed15"}
Oct 30 19:57:49.850008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.850012    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"
Oct 30 19:57:49.850569 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:49.850063    5704 scope.go:122] "RemoveContainer" containerID="2fe03da0147966fb1e6b653c0505b5ffa0d0d0945e691ea376ba312f0aaf31a5"
Oct 30 19:57:50.071938 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:50.071901    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"]
Oct 30 19:57:50.116781 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:50.116716    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-db06cf-openshift-default-5c5ff5d79f-z7drp"]
Oct 30 19:57:50.292929 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:50.292893    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:50.495148 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:50.495066    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:50.840863 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:50.840829    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:51.179935 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:51.179856    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" path="/var/lib/kubelet/pods/8f538c70-f8bf-4a0a-bcc0-6800a8b495ce/volumes"
Oct 30 19:57:52.574800 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:52.574756    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:53.284580 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:53.284542    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:54.722714 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:54.722676    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:54.945831 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:54.945804    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"]
Oct 30 19:57:55.407612 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:55.407580    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:55.729399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:55.729319    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:57.755728 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:57.755671    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:59.659124 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:59.659087    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:59.905501 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:59.905458    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:57:59.946284 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:57:59.946202    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64" podUID="0f628790-151a-4760-9dfd-62f0ca9c87e4" containerName="payload-processing" containerID="cri-o://2c5e10ea1c27d6acbadb8fb6da10aa670bdaf415e9e6b6ed34972fa193c067f3" gracePeriod=25
Oct 30 19:58:02.307234 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.307201    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"]
Oct 30 19:58:02.307721 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.307472    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb" podUID="479084c0-041e-4bf9-b2ce-0721d9fb3f75" containerName="maas-api" containerID="cri-o://5228733e421ea710909e671e8b0a643b01146085a3884967ab164cbc7e56bcba" gracePeriod=30
Oct 30 19:58:02.422586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.422554    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:02.597643 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.597614    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:58:02.672622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.672589    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-metrics-certs\" (UniqueName: \"kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-metrics-certs\") pod \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") "
Oct 30 19:58:02.672791 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.672688    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/479084c0-041e-4bf9-b2ce-0721d9fb3f75-kube-api-access-ztbrj\" (UniqueName: \"kubernetes.io/projected/479084c0-041e-4bf9-b2ce-0721d9fb3f75-kube-api-access-ztbrj\") pod \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") "
Oct 30 19:58:02.672855 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.672807    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-maas-api-tls\") pod \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") "
Oct 30 19:58:02.672855 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.672843    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/479084c0-041e-4bf9-b2ce-0721d9fb3f75-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/479084c0-041e-4bf9-b2ce-0721d9fb3f75-openshift-service-ca\") pod \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\" (UID: \"479084c0-041e-4bf9-b2ce-0721d9fb3f75\") "
Oct 30 19:58:02.673301 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.673273    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/479084c0-041e-4bf9-b2ce-0721d9fb3f75-openshift-service-ca" pod "479084c0-041e-4bf9-b2ce-0721d9fb3f75" (UID: "479084c0-041e-4bf9-b2ce-0721d9fb3f75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:02.674739 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.674711    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-metrics-certs" pod "479084c0-041e-4bf9-b2ce-0721d9fb3f75" (UID: "479084c0-041e-4bf9-b2ce-0721d9fb3f75"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:02.674845 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.674801    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/479084c0-041e-4bf9-b2ce-0721d9fb3f75-kube-api-access-ztbrj" pod "479084c0-041e-4bf9-b2ce-0721d9fb3f75" (UID: "479084c0-041e-4bf9-b2ce-0721d9fb3f75"). InnerVolumeSpecName "kube-api-access-ztbrj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:02.674896 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.674851    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-maas-api-tls" pod "479084c0-041e-4bf9-b2ce-0721d9fb3f75" (UID: "479084c0-041e-4bf9-b2ce-0721d9fb3f75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:02.774983 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.774939    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztbrj\" (UniqueName: \"kubernetes.io/projected/479084c0-041e-4bf9-b2ce-0721d9fb3f75-kube-api-access-ztbrj\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:02.774983 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.774980    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:02.774983 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.774992    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/479084c0-041e-4bf9-b2ce-0721d9fb3f75-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:02.774983 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.775002    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/479084c0-041e-4bf9-b2ce-0721d9fb3f75-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:02.887774 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.887685    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:02.928434 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.928405    5704 generic.go:356] "Generic (PLEG): container finished" podID="479084c0-041e-4bf9-b2ce-0721d9fb3f75" containerID="5228733e421ea710909e671e8b0a643b01146085a3884967ab164cbc7e56bcba" exitCode=0
Oct 30 19:58:02.928618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.928450    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"
Oct 30 19:58:02.928618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.928477    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb" event={"ID":"479084c0-041e-4bf9-b2ce-0721d9fb3f75","Type":"ContainerDied","Data":"5228733e421ea710909e671e8b0a643b01146085a3884967ab164cbc7e56bcba"}
Oct 30 19:58:02.928618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.928517    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb" event={"ID":"479084c0-041e-4bf9-b2ce-0721d9fb3f75","Type":"ContainerDied","Data":"0a7d744065bb0bd2ec37e8cc5605b0c2a1d0998229c1daf85cf774fac363bd4c"}
Oct 30 19:58:02.928618 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.928538    5704 scope.go:122] "RemoveContainer" containerID="5228733e421ea710909e671e8b0a643b01146085a3884967ab164cbc7e56bcba"
Oct 30 19:58:02.951812 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.951603    5704 scope.go:122] "RemoveContainer" containerID="5228733e421ea710909e671e8b0a643b01146085a3884967ab164cbc7e56bcba"
Oct 30 19:58:02.952501 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:58:02.952464    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5228733e421ea710909e671e8b0a643b01146085a3884967ab164cbc7e56bcba\": container with ID starting with 5228733e421ea710909e671e8b0a643b01146085a3884967ab164cbc7e56bcba not found: ID does not exist" containerID="5228733e421ea710909e671e8b0a643b01146085a3884967ab164cbc7e56bcba"
Oct 30 19:58:02.952669 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:02.952506    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5228733e421ea710909e671e8b0a643b01146085a3884967ab164cbc7e56bcba"} err="failed to get container status \"5228733e421ea710909e671e8b0a643b01146085a3884967ab164cbc7e56bcba\": rpc error: code = NotFound desc = could not find container \"5228733e421ea710909e671e8b0a643b01146085a3884967ab164cbc7e56bcba\": container with ID starting with 5228733e421ea710909e671e8b0a643b01146085a3884967ab164cbc7e56bcba not found: ID does not exist"
Oct 30 19:58:03.089059 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.088989    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:03.102263 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.101205    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"]
Oct 30 19:58:03.106877 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.106844    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-b2604c-649b9f4d8b-mnnpb"]
Oct 30 19:58:03.183493 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.183403    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="479084c0-041e-4bf9-b2ce-0721d9fb3f75" path="/var/lib/kubelet/pods/479084c0-041e-4bf9-b2ce-0721d9fb3f75/volumes"
Oct 30 19:58:03.284886 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.284851    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"]
Oct 30 19:58:03.285384 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285371    5704 cpu_manager.go:414] "containerMap: removing container" podUID="0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9" containerName="payload-processing"
Oct 30 19:58:03.285428 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285386    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9" containerName="payload-processing"
Oct 30 19:58:03.285428 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285401    5704 cpu_manager.go:414] "containerMap: removing container" podUID="6fdbaa8d-ad3f-4202-9ddc-e208538a14ac" containerName="payload-processing"
Oct 30 19:58:03.285428 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285406    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fdbaa8d-ad3f-4202-9ddc-e208538a14ac" containerName="payload-processing"
Oct 30 19:58:03.285428 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285415    5704 cpu_manager.go:414] "containerMap: removing container" podUID="faa9d942-3670-49ef-b4bc-a3fe995a642f" containerName="maas-api"
Oct 30 19:58:03.285428 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285421    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="faa9d942-3670-49ef-b4bc-a3fe995a642f" containerName="maas-api"
Oct 30 19:58:03.285428 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285429    5704 cpu_manager.go:414] "containerMap: removing container" podUID="2f63052b-ba80-4c6e-9a7c-4413eebb990b" containerName="maas-api"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285434    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2f63052b-ba80-4c6e-9a7c-4413eebb990b" containerName="maas-api"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285446    5704 cpu_manager.go:414] "containerMap: removing container" podUID="479084c0-041e-4bf9-b2ce-0721d9fb3f75" containerName="maas-api"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285450    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="479084c0-041e-4bf9-b2ce-0721d9fb3f75" containerName="maas-api"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285462    5704 cpu_manager.go:414] "containerMap: removing container" podUID="1daee16d-a4f6-4f87-b792-b3866450b2b2" containerName="payload-processing"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285467    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1daee16d-a4f6-4f87-b792-b3866450b2b2" containerName="payload-processing"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285473    5704 cpu_manager.go:414] "containerMap: removing container" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerName="istio-proxy"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285478    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerName="istio-proxy"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285496    5704 cpu_manager.go:414] "containerMap: removing container" podUID="fa6c415c-9fee-4b1e-979f-5764d5cba9a4" containerName="istio-proxy"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285500    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa6c415c-9fee-4b1e-979f-5764d5cba9a4" containerName="istio-proxy"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285508    5704 cpu_manager.go:414] "containerMap: removing container" podUID="f9b0ac2e-2d36-454f-8666-25acd2696603" containerName="payload-processing"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285512    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f9b0ac2e-2d36-454f-8666-25acd2696603" containerName="payload-processing"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285525    5704 cpu_manager.go:414] "containerMap: removing container" podUID="f67a70de-5e78-4a06-a16b-cd846189120d" containerName="payload-pre-processing"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285529    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f67a70de-5e78-4a06-a16b-cd846189120d" containerName="payload-pre-processing"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285540    5704 cpu_manager.go:414] "containerMap: removing container" podUID="622d8410-59c1-4a09-acce-2aae15528bac" containerName="maas-api"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285544    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="622d8410-59c1-4a09-acce-2aae15528bac" containerName="maas-api"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285554    5704 cpu_manager.go:414] "containerMap: removing container" podUID="af934c90-d9b7-427d-acce-67f9cc332a82" containerName="istio-proxy"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285558    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="af934c90-d9b7-427d-acce-67f9cc332a82" containerName="istio-proxy"
Oct 30 19:58:03.285622 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285624    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="0e21e91f-d5eb-4e2b-8ae8-5b47a22732f9" containerName="payload-processing"
Oct 30 19:58:03.286243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285632    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fdbaa8d-ad3f-4202-9ddc-e208538a14ac" containerName="payload-processing"
Oct 30 19:58:03.286243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285640    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="faa9d942-3670-49ef-b4bc-a3fe995a642f" containerName="maas-api"
Oct 30 19:58:03.286243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285645    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="2f63052b-ba80-4c6e-9a7c-4413eebb990b" containerName="maas-api"
Oct 30 19:58:03.286243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285652    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="479084c0-041e-4bf9-b2ce-0721d9fb3f75" containerName="maas-api"
Oct 30 19:58:03.286243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285659    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="1daee16d-a4f6-4f87-b792-b3866450b2b2" containerName="payload-processing"
Oct 30 19:58:03.286243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285665    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerName="istio-proxy"
Oct 30 19:58:03.286243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285673    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa6c415c-9fee-4b1e-979f-5764d5cba9a4" containerName="istio-proxy"
Oct 30 19:58:03.286243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285679    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="f9b0ac2e-2d36-454f-8666-25acd2696603" containerName="payload-processing"
Oct 30 19:58:03.286243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285685    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerName="istio-proxy"
Oct 30 19:58:03.286243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285691    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="f67a70de-5e78-4a06-a16b-cd846189120d" containerName="payload-pre-processing"
Oct 30 19:58:03.286243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285698    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="622d8410-59c1-4a09-acce-2aae15528bac" containerName="maas-api"
Oct 30 19:58:03.286243 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.285704    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="af934c90-d9b7-427d-acce-67f9cc332a82" containerName="istio-proxy"
Oct 30 19:58:03.290963 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.290941    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"
Oct 30 19:58:03.340346 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.340323    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-extmod-b-f77d05-dockercfg-cqfxk\""
Oct 30 19:58:03.340689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.340385    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-extmod-b-f77d05\""
Oct 30 19:58:03.349311 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.349286    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"]
Oct 30 19:58:03.385246 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.385219    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55j4d\" (UniqueName: \"kubernetes.io/projected/53821e33-7446-47cd-afd1-7489845cda73-kube-api-access-55j4d\") pod \"payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp\" (UID: \"53821e33-7446-47cd-afd1-7489845cda73\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"
Oct 30 19:58:03.385397 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.385307    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53821e33-7446-47cd-afd1-7489845cda73-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp\" (UID: \"53821e33-7446-47cd-afd1-7489845cda73\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"
Oct 30 19:58:03.385397 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:58:03.385369    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:58:03.446860 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.446773    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"]
Oct 30 19:58:03.458771 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.458736    5704 cpu_manager.go:414] "containerMap: removing container" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerName="istio-proxy"
Oct 30 19:58:03.459017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.458997    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8f538c70-f8bf-4a0a-bcc0-6800a8b495ce" containerName="istio-proxy"
Oct 30 19:58:03.465132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.463694    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"
Oct 30 19:58:03.486329 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.486293    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55j4d\" (UniqueName: \"kubernetes.io/projected/53821e33-7446-47cd-afd1-7489845cda73-kube-api-access-55j4d\") pod \"payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp\" (UID: \"53821e33-7446-47cd-afd1-7489845cda73\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"
Oct 30 19:58:03.486492 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.486340    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d4049f4-af8d-407e-b344-b8730a24b4e2-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt\" (UID: \"7d4049f4-af8d-407e-b344-b8730a24b4e2\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"
Oct 30 19:58:03.486587 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.486515    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpnk5\" (UniqueName: \"kubernetes.io/projected/7d4049f4-af8d-407e-b344-b8730a24b4e2-kube-api-access-vpnk5\") pod \"payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt\" (UID: \"7d4049f4-af8d-407e-b344-b8730a24b4e2\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"
Oct 30 19:58:03.486650 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.486610    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53821e33-7446-47cd-afd1-7489845cda73-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp\" (UID: \"53821e33-7446-47cd-afd1-7489845cda73\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"
Oct 30 19:58:03.487449 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.487421    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53821e33-7446-47cd-afd1-7489845cda73-plugins-config-volume\") pod \"payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp\" (UID: \"53821e33-7446-47cd-afd1-7489845cda73\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"
Oct 30 19:58:03.507138 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.507115    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"]
Oct 30 19:58:03.545787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.545754    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55j4d\" (UniqueName: \"kubernetes.io/projected/53821e33-7446-47cd-afd1-7489845cda73-kube-api-access-55j4d\") pod \"payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp\" (UID: \"53821e33-7446-47cd-afd1-7489845cda73\") " pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"
Oct 30 19:58:03.587522 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.587495    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d4049f4-af8d-407e-b344-b8730a24b4e2-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt\" (UID: \"7d4049f4-af8d-407e-b344-b8730a24b4e2\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"
Oct 30 19:58:03.587723 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.587619    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpnk5\" (UniqueName: \"kubernetes.io/projected/7d4049f4-af8d-407e-b344-b8730a24b4e2-kube-api-access-vpnk5\") pod \"payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt\" (UID: \"7d4049f4-af8d-407e-b344-b8730a24b4e2\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"
Oct 30 19:58:03.588298 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.588277    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d4049f4-af8d-407e-b344-b8730a24b4e2-plugins-config-volume\") pod \"payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt\" (UID: \"7d4049f4-af8d-407e-b344-b8730a24b4e2\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"
Oct 30 19:58:03.604784 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.604761    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"
Oct 30 19:58:03.654065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.654015    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpnk5\" (UniqueName: \"kubernetes.io/projected/7d4049f4-af8d-407e-b344-b8730a24b4e2-kube-api-access-vpnk5\") pod \"payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt\" (UID: \"7d4049f4-af8d-407e-b344-b8730a24b4e2\") " pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"
Oct 30 19:58:03.683742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.683710    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:03.787182 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.787142    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"
Oct 30 19:58:03.977189 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:03.977155    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"]
Oct 30 19:58:03.980859 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:58:03.980824    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53821e33_7446_47cd_afd1_7489845cda73.slice/crio-90148cccd46d391ab8dcbc8872ba80016a528ddb91c3d668ca160040441ecf0c WatchSource:0}: Error finding container 90148cccd46d391ab8dcbc8872ba80016a528ddb91c3d668ca160040441ecf0c: Status 404 returned error can't find the container with id 90148cccd46d391ab8dcbc8872ba80016a528ddb91c3d668ca160040441ecf0c
Oct 30 19:58:04.037369 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:04.037317    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:04.159335 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:04.159312    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"]
Oct 30 19:58:04.161421 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:58:04.161397    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d4049f4_af8d_407e_b344_b8730a24b4e2.slice/crio-5d9b93f2f243ee752b91610e2c20f70cf81f4e44230a91601262a85befad379b WatchSource:0}: Error finding container 5d9b93f2f243ee752b91610e2c20f70cf81f4e44230a91601262a85befad379b: Status 404 returned error can't find the container with id 5d9b93f2f243ee752b91610e2c20f70cf81f4e44230a91601262a85befad379b
Oct 30 19:58:04.343961 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:04.343892    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:04.965363 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:04.965328    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp" event={"ID":"53821e33-7446-47cd-afd1-7489845cda73","Type":"ContainerStarted","Data":"70ddd254418a7bc7001052793c151c32b171e82af6c9b34df8cc403928dc9440"}
Oct 30 19:58:04.965554 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:04.965392    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"
Oct 30 19:58:04.965554 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:04.965416    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp" event={"ID":"53821e33-7446-47cd-afd1-7489845cda73","Type":"ContainerStarted","Data":"90148cccd46d391ab8dcbc8872ba80016a528ddb91c3d668ca160040441ecf0c"}
Oct 30 19:58:04.966956 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:04.966925    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt" event={"ID":"7d4049f4-af8d-407e-b344-b8730a24b4e2","Type":"ContainerStarted","Data":"3817b8a5739f04748945a8c737ab43ef2a715e3c67401e8a5463739e5581fa5d"}
Oct 30 19:58:04.967107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:04.966999    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt" event={"ID":"7d4049f4-af8d-407e-b344-b8730a24b4e2","Type":"ContainerStarted","Data":"5d9b93f2f243ee752b91610e2c20f70cf81f4e44230a91601262a85befad379b"}
Oct 30 19:58:04.967107 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:04.967022    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"
Oct 30 19:58:04.985289 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:04.985259    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:05.078327 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:05.078276    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp" podStartSLOduration=2.078262021 podStartE2EDuration="2.078262021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:58:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:58:05.076539368 +0000 UTC m=+4258.022707285" watchObservedRunningTime="2026-10-30 19:58:05.078262021 +0000 UTC m=+4258.024429923"
Oct 30 19:58:05.190442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:05.189954    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt" podStartSLOduration=2.18993636 podStartE2EDuration="2.18993636s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:58:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:58:05.187125522 +0000 UTC m=+4258.133293422" watchObservedRunningTime="2026-10-30 19:58:05.18993636 +0000 UTC m=+4258.136104278"
Oct 30 19:58:05.322672 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:05.322642    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:05.570661 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:05.570630    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:05.813795 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:05.813738    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:06.324273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:06.324233    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:06.506948 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:06.506908    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64" podUID="0f628790-151a-4760-9dfd-62f0ca9c87e4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:06.673854 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:06.673772    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:06.853472 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:06.853419    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:09.939561 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:09.939522    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:11.554739 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:11.554706    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:11.844695 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:11.844596    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:12.047529 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:12.047494    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:12.245452 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:12.245373    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:12.578948 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:12.578855    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:12.911879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:12.911795    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:13.154471 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:13.154437    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:14.571545 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:14.571512    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:14.803329 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:14.803292    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:15.019377 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:15.019346    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:15.976051 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:15.976011    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"
Oct 30 19:58:15.976428 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:15.976098    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"
Oct 30 19:58:16.506572 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:16.506541    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64" podUID="0f628790-151a-4760-9dfd-62f0ca9c87e4" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:17.054542 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:17.054507    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:18.153269 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:18.153232    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:18.341379 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:18.341341    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:18.774159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:18.774124    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:58:19.963244 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:19.963206    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:20.203079 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:20.203013    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"
Oct 30 19:58:20.217532 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:20.217507    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:20.291214 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:20.291187    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0f628790-151a-4760-9dfd-62f0ca9c87e4-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f628790-151a-4760-9dfd-62f0ca9c87e4-plugins-config-volume\") pod \"0f628790-151a-4760-9dfd-62f0ca9c87e4\" (UID: \"0f628790-151a-4760-9dfd-62f0ca9c87e4\") "
Oct 30 19:58:20.291443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:20.291417    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0f628790-151a-4760-9dfd-62f0ca9c87e4-kube-api-access-5zcmk\" (UniqueName: \"kubernetes.io/projected/0f628790-151a-4760-9dfd-62f0ca9c87e4-kube-api-access-5zcmk\") pod \"0f628790-151a-4760-9dfd-62f0ca9c87e4\" (UID: \"0f628790-151a-4760-9dfd-62f0ca9c87e4\") "
Oct 30 19:58:20.291631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:20.291606    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0f628790-151a-4760-9dfd-62f0ca9c87e4-plugins-config-volume" pod "0f628790-151a-4760-9dfd-62f0ca9c87e4" (UID: "0f628790-151a-4760-9dfd-62f0ca9c87e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:20.292097 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:20.292073    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0f628790-151a-4760-9dfd-62f0ca9c87e4-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:20.294385 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:20.294347    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0f628790-151a-4760-9dfd-62f0ca9c87e4-kube-api-access-5zcmk" pod "0f628790-151a-4760-9dfd-62f0ca9c87e4" (UID: "0f628790-151a-4760-9dfd-62f0ca9c87e4"). InnerVolumeSpecName "kube-api-access-5zcmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:20.393042 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:20.392998    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5zcmk\" (UniqueName: \"kubernetes.io/projected/0f628790-151a-4760-9dfd-62f0ca9c87e4-kube-api-access-5zcmk\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:20.466215 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:20.466103    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:20.895670 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:20.895631    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:21.069623 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:21.069590    5704 generic.go:356] "Generic (PLEG): container finished" podID="0f628790-151a-4760-9dfd-62f0ca9c87e4" containerID="2c5e10ea1c27d6acbadb8fb6da10aa670bdaf415e9e6b6ed34972fa193c067f3" exitCode=0
Oct 30 19:58:21.070106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:21.069636    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"
Oct 30 19:58:21.070106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:21.069652    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64" event={"ID":"0f628790-151a-4760-9dfd-62f0ca9c87e4","Type":"ContainerDied","Data":"2c5e10ea1c27d6acbadb8fb6da10aa670bdaf415e9e6b6ed34972fa193c067f3"}
Oct 30 19:58:21.070106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:21.069704    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64" event={"ID":"0f628790-151a-4760-9dfd-62f0ca9c87e4","Type":"ContainerDied","Data":"962f7954f9a94f1ed1ec3f79c8ecdc84458d97da2e7325cbf52176cee9bf2f08"}
Oct 30 19:58:21.070106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:21.069735    5704 scope.go:122] "RemoveContainer" containerID="2c5e10ea1c27d6acbadb8fb6da10aa670bdaf415e9e6b6ed34972fa193c067f3"
Oct 30 19:58:21.079496 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:21.079481    5704 scope.go:122] "RemoveContainer" containerID="2c5e10ea1c27d6acbadb8fb6da10aa670bdaf415e9e6b6ed34972fa193c067f3"
Oct 30 19:58:21.079753 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:58:21.079734    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c5e10ea1c27d6acbadb8fb6da10aa670bdaf415e9e6b6ed34972fa193c067f3\": container with ID starting with 2c5e10ea1c27d6acbadb8fb6da10aa670bdaf415e9e6b6ed34972fa193c067f3 not found: ID does not exist" containerID="2c5e10ea1c27d6acbadb8fb6da10aa670bdaf415e9e6b6ed34972fa193c067f3"
Oct 30 19:58:21.079794 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:21.079761    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c5e10ea1c27d6acbadb8fb6da10aa670bdaf415e9e6b6ed34972fa193c067f3"} err="failed to get container status \"2c5e10ea1c27d6acbadb8fb6da10aa670bdaf415e9e6b6ed34972fa193c067f3\": rpc error: code = NotFound desc = could not find container \"2c5e10ea1c27d6acbadb8fb6da10aa670bdaf415e9e6b6ed34972fa193c067f3\": container with ID starting with 2c5e10ea1c27d6acbadb8fb6da10aa670bdaf415e9e6b6ed34972fa193c067f3 not found: ID does not exist"
Oct 30 19:58:21.222841 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:21.222775    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"]
Oct 30 19:58:21.269053 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:21.269007    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w3-b2604c-76764c7858-tww64"]
Oct 30 19:58:22.400609 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:22.400536    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:23.179049 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:23.179001    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0f628790-151a-4760-9dfd-62f0ca9c87e4" path="/var/lib/kubelet/pods/0f628790-151a-4760-9dfd-62f0ca9c87e4/volumes"
Oct 30 19:58:24.905065 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:24.904999    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"]
Oct 30 19:58:25.011966 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:25.011880    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"]
Oct 30 19:58:25.286017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:25.285981    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:25.619170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:25.619095    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:25.963441 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:25.963343    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:26.867117 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:26.867079    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:27.357572 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:27.357537    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"]
Oct 30 19:58:27.475742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:27.475708    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"]
Oct 30 19:58:27.750192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:27.750159    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:27.939970 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:27.939928    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:28.599506 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:28.599475    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:28.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:28.853628    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:29.106091 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:29.105986    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:29.604377 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:29.604341    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:29.905619 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:29.905530    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7" podUID="1803662c-638a-4442-b019-1ef97209f2a8" containerName="payload-processing" containerID="cri-o://e6add73921011ae7a832f5fdd6d4ed385ac721e46742ec325bb1ac09f119e13f" gracePeriod=25
Oct 30 19:58:30.012630 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:30.012593    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx" podUID="57bc5590-a521-4cdd-9f55-7667b13bcfba" containerName="payload-pre-processing" containerID="cri-o://767c04fb670b3932880fb0df8bd509adc25bdb809b40e6eba68771abd3b3f138" gracePeriod=25
Oct 30 19:58:32.358688 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:32.358649    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2" podUID="20d434f7-33e5-4136-9114-43d281037737" containerName="payload-processing" containerID="cri-o://a21ab7d115bde8c9803c85a09ccc758064afceeff6b7caf01d2551950e1c9b8f" gracePeriod=25
Oct 30 19:58:32.476288 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:32.476253    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62" podUID="f1c467c6-c1dc-4095-9849-35acde68f3a3" containerName="payload-pre-processing" containerID="cri-o://9d88f0bd77097c3a6120097a075cd90341af362201886527149c18ad8637910d" gracePeriod=25
Oct 30 19:58:32.937508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:32.937476    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"]
Oct 30 19:58:33.213878 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:33.213801    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:33.384112 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:58:33.384075    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:58:33.458589 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:33.458552    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:34.129933 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:34.129897    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:36.021029 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:36.020998    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx" podUID="57bc5590-a521-4cdd-9f55-7667b13bcfba" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:36.021443 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:36.021074    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7" podUID="1803662c-638a-4442-b019-1ef97209f2a8" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:36.342577 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:36.342486    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:36.578060 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:36.578015    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:36.713938 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:36.713864    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"]
Oct 30 19:58:36.714117 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:36.714096    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp" podUID="902a14d3-7e9d-448c-a951-465010109b2c" containerName="istio-proxy" containerID="cri-o://fb8946f6638ff867099866d0b9b23efaf0535cbfa40c1d261356d606f9e53291" gracePeriod=30
Oct 30 19:58:36.771521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:36.771494    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:37.240010 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:37.239977    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:37.749419 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:37.749384    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62" podUID="f1c467c6-c1dc-4095-9849-35acde68f3a3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:37.749620 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:37.749498    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2" podUID="20d434f7-33e5-4136-9114-43d281037737" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:37.938674 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:37.938639    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx" podUID="17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b" containerName="payload-pre-processing" containerID="cri-o://8dd731f7e3035379faac95addd792db5037833e98f177fa21d9abec395fd9800" gracePeriod=25
Oct 30 19:58:39.082898 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:39.081668    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:40.988407 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:40.988368    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx" podUID="17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:41.974609 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:41.974587    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:58:42.048369 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.048342    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/902a14d3-7e9d-448c-a951-465010109b2c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/902a14d3-7e9d-448c-a951-465010109b2c-istiod-ca-cert\") pod \"902a14d3-7e9d-448c-a951-465010109b2c\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") "
Oct 30 19:58:42.048745 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.048387    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-envoy\") pod \"902a14d3-7e9d-448c-a951-465010109b2c\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") "
Oct 30 19:58:42.048745 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.048433    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-kube-api-access-zcgtm\" (UniqueName: \"kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-kube-api-access-zcgtm\") pod \"902a14d3-7e9d-448c-a951-465010109b2c\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") "
Oct 30 19:58:42.048745 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.048453    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/902a14d3-7e9d-448c-a951-465010109b2c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/902a14d3-7e9d-448c-a951-465010109b2c-istio-podinfo\") pod \"902a14d3-7e9d-448c-a951-465010109b2c\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") "
Oct 30 19:58:42.048745 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.048486    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-data\") pod \"902a14d3-7e9d-448c-a951-465010109b2c\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") "
Oct 30 19:58:42.048745 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.048506    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-certs\") pod \"902a14d3-7e9d-448c-a951-465010109b2c\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") "
Oct 30 19:58:42.048745 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.048644    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-istio-token\" (UniqueName: \"kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-istio-token\") pod \"902a14d3-7e9d-448c-a951-465010109b2c\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") "
Oct 30 19:58:42.048745 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.048717    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-socket\") pod \"902a14d3-7e9d-448c-a951-465010109b2c\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") "
Oct 30 19:58:42.049216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.048752    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-credential-socket\") pod \"902a14d3-7e9d-448c-a951-465010109b2c\" (UID: \"902a14d3-7e9d-448c-a951-465010109b2c\") "
Oct 30 19:58:42.049216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.048813    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-data" pod "902a14d3-7e9d-448c-a951-465010109b2c" (UID: "902a14d3-7e9d-448c-a951-465010109b2c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:42.049216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.048836    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/902a14d3-7e9d-448c-a951-465010109b2c-istiod-ca-cert" pod "902a14d3-7e9d-448c-a951-465010109b2c" (UID: "902a14d3-7e9d-448c-a951-465010109b2c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:42.049216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.049060    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-socket" pod "902a14d3-7e9d-448c-a951-465010109b2c" (UID: "902a14d3-7e9d-448c-a951-465010109b2c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:42.049216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.049154    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-credential-socket" pod "902a14d3-7e9d-448c-a951-465010109b2c" (UID: "902a14d3-7e9d-448c-a951-465010109b2c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:42.049216 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.049166    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-certs" pod "902a14d3-7e9d-448c-a951-465010109b2c" (UID: "902a14d3-7e9d-448c-a951-465010109b2c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:42.049556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.049380    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:42.049556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.049401    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:42.049556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.049415    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/902a14d3-7e9d-448c-a951-465010109b2c-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:42.049556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.049430    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:42.049556 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.049443    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:42.050723 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.050699    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-kube-api-access-zcgtm" pod "902a14d3-7e9d-448c-a951-465010109b2c" (UID: "902a14d3-7e9d-448c-a951-465010109b2c"). InnerVolumeSpecName "kube-api-access-zcgtm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:42.050852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.050757    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/902a14d3-7e9d-448c-a951-465010109b2c-istio-podinfo" pod "902a14d3-7e9d-448c-a951-465010109b2c" (UID: "902a14d3-7e9d-448c-a951-465010109b2c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:58:42.050852 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.050772    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-envoy" pod "902a14d3-7e9d-448c-a951-465010109b2c" (UID: "902a14d3-7e9d-448c-a951-465010109b2c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:42.050933 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.050862    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-istio-token" pod "902a14d3-7e9d-448c-a951-465010109b2c" (UID: "902a14d3-7e9d-448c-a951-465010109b2c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:42.150602 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.150575    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/902a14d3-7e9d-448c-a951-465010109b2c-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:42.150602 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.150602    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zcgtm\" (UniqueName: \"kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-kube-api-access-zcgtm\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:42.150806 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.150612    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/902a14d3-7e9d-448c-a951-465010109b2c-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:42.150806 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.150622    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/902a14d3-7e9d-448c-a951-465010109b2c-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:42.187360 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.187334    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"]
Oct 30 19:58:42.189322 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.189270    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"]
Oct 30 19:58:42.192387 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.192360    5704 generic.go:356] "Generic (PLEG): container finished" podID="902a14d3-7e9d-448c-a951-465010109b2c" containerID="fb8946f6638ff867099866d0b9b23efaf0535cbfa40c1d261356d606f9e53291" exitCode=0
Oct 30 19:58:42.192505 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.192402    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"
Oct 30 19:58:42.192505 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.192443    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp" event={"ID":"902a14d3-7e9d-448c-a951-465010109b2c","Type":"ContainerDied","Data":"fb8946f6638ff867099866d0b9b23efaf0535cbfa40c1d261356d606f9e53291"}
Oct 30 19:58:42.192632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.192506    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp" event={"ID":"902a14d3-7e9d-448c-a951-465010109b2c","Type":"ContainerDied","Data":"9a78a34269a52a4fa652ad1b6641e462205bb1b52699cab16ce40fb491e2dace"}
Oct 30 19:58:42.192632 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.192539    5704 scope.go:122] "RemoveContainer" containerID="fb8946f6638ff867099866d0b9b23efaf0535cbfa40c1d261356d606f9e53291"
Oct 30 19:58:42.204422 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.204406    5704 scope.go:122] "RemoveContainer" containerID="12164d02a5113750f6074c2c1120775156b0f0514e8ddd80b85757c005dfe0eb"
Oct 30 19:58:42.211170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.211150    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"]
Oct 30 19:58:42.215499 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.215483    5704 scope.go:122] "RemoveContainer" containerID="fb8946f6638ff867099866d0b9b23efaf0535cbfa40c1d261356d606f9e53291"
Oct 30 19:58:42.215748 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:58:42.215727    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb8946f6638ff867099866d0b9b23efaf0535cbfa40c1d261356d606f9e53291\": container with ID starting with fb8946f6638ff867099866d0b9b23efaf0535cbfa40c1d261356d606f9e53291 not found: ID does not exist" containerID="fb8946f6638ff867099866d0b9b23efaf0535cbfa40c1d261356d606f9e53291"
Oct 30 19:58:42.215830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.215761    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb8946f6638ff867099866d0b9b23efaf0535cbfa40c1d261356d606f9e53291"} err="failed to get container status \"fb8946f6638ff867099866d0b9b23efaf0535cbfa40c1d261356d606f9e53291\": rpc error: code = NotFound desc = could not find container \"fb8946f6638ff867099866d0b9b23efaf0535cbfa40c1d261356d606f9e53291\": container with ID starting with fb8946f6638ff867099866d0b9b23efaf0535cbfa40c1d261356d606f9e53291 not found: ID does not exist"
Oct 30 19:58:42.215830 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.215784    5704 scope.go:122] "RemoveContainer" containerID="12164d02a5113750f6074c2c1120775156b0f0514e8ddd80b85757c005dfe0eb"
Oct 30 19:58:42.216087 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:58:42.216063    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"12164d02a5113750f6074c2c1120775156b0f0514e8ddd80b85757c005dfe0eb\": container with ID starting with 12164d02a5113750f6074c2c1120775156b0f0514e8ddd80b85757c005dfe0eb not found: ID does not exist" containerID="12164d02a5113750f6074c2c1120775156b0f0514e8ddd80b85757c005dfe0eb"
Oct 30 19:58:42.216133 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.216096    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"12164d02a5113750f6074c2c1120775156b0f0514e8ddd80b85757c005dfe0eb"} err="failed to get container status \"12164d02a5113750f6074c2c1120775156b0f0514e8ddd80b85757c005dfe0eb\": rpc error: code = NotFound desc = could not find container \"12164d02a5113750f6074c2c1120775156b0f0514e8ddd80b85757c005dfe0eb\": container with ID starting with 12164d02a5113750f6074c2c1120775156b0f0514e8ddd80b85757c005dfe0eb not found: ID does not exist"
Oct 30 19:58:42.344230 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.344154    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:42.520914 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.520884    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"]
Oct 30 19:58:42.524145 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:42.524122    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w0-d6f58f-openshift-default-755c9c9767-h9nmp"]
Oct 30 19:58:43.187131 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:43.187102    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="902a14d3-7e9d-448c-a951-465010109b2c" path="/var/lib/kubelet/pods/902a14d3-7e9d-448c-a951-465010109b2c/volumes"
Oct 30 19:58:44.038793 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:44.038741    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:46.020900 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:46.020867    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7" podUID="1803662c-638a-4442-b019-1ef97209f2a8" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:46.021270 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:46.020867    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx" podUID="57bc5590-a521-4cdd-9f55-7667b13bcfba" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:47.748691 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:47.748647    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2" podUID="20d434f7-33e5-4136-9114-43d281037737" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:47.749196 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:47.748770    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62" podUID="f1c467c6-c1dc-4095-9849-35acde68f3a3" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:49.076528 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:49.076488    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:49.338461 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:49.338367    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:50.167903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.167839    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"
Oct 30 19:58:50.232292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.232260    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1803662c-638a-4442-b019-1ef97209f2a8-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1803662c-638a-4442-b019-1ef97209f2a8-plugins-config-volume\") pod \"1803662c-638a-4442-b019-1ef97209f2a8\" (UID: \"1803662c-638a-4442-b019-1ef97209f2a8\") "
Oct 30 19:58:50.232538 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.232516    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1803662c-638a-4442-b019-1ef97209f2a8-kube-api-access-hz6xf\" (UniqueName: \"kubernetes.io/projected/1803662c-638a-4442-b019-1ef97209f2a8-kube-api-access-hz6xf\") pod \"1803662c-638a-4442-b019-1ef97209f2a8\" (UID: \"1803662c-638a-4442-b019-1ef97209f2a8\") "
Oct 30 19:58:50.232768 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.232747    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1803662c-638a-4442-b019-1ef97209f2a8-plugins-config-volume" pod "1803662c-638a-4442-b019-1ef97209f2a8" (UID: "1803662c-638a-4442-b019-1ef97209f2a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:50.232985 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.232967    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1803662c-638a-4442-b019-1ef97209f2a8-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:50.234770 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.234745    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1803662c-638a-4442-b019-1ef97209f2a8-kube-api-access-hz6xf" pod "1803662c-638a-4442-b019-1ef97209f2a8" (UID: "1803662c-638a-4442-b019-1ef97209f2a8"). InnerVolumeSpecName "kube-api-access-hz6xf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:50.238531 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.238498    5704 generic.go:356] "Generic (PLEG): container finished" podID="1803662c-638a-4442-b019-1ef97209f2a8" containerID="e6add73921011ae7a832f5fdd6d4ed385ac721e46742ec325bb1ac09f119e13f" exitCode=0
Oct 30 19:58:50.238619 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.238554    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"
Oct 30 19:58:50.238694 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.238674    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7" event={"ID":"1803662c-638a-4442-b019-1ef97209f2a8","Type":"ContainerDied","Data":"e6add73921011ae7a832f5fdd6d4ed385ac721e46742ec325bb1ac09f119e13f"}
Oct 30 19:58:50.238759 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.238727    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7" event={"ID":"1803662c-638a-4442-b019-1ef97209f2a8","Type":"ContainerDied","Data":"48065517b584cb81d385fcaf623f4503e0ee0c71fd02abc6398a122aadd8e5d4"}
Oct 30 19:58:50.238811 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.238762    5704 scope.go:122] "RemoveContainer" containerID="e6add73921011ae7a832f5fdd6d4ed385ac721e46742ec325bb1ac09f119e13f"
Oct 30 19:58:50.241282 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.241257    5704 generic.go:356] "Generic (PLEG): container finished" podID="57bc5590-a521-4cdd-9f55-7667b13bcfba" containerID="767c04fb670b3932880fb0df8bd509adc25bdb809b40e6eba68771abd3b3f138" exitCode=0
Oct 30 19:58:50.241399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.241331    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx" event={"ID":"57bc5590-a521-4cdd-9f55-7667b13bcfba","Type":"ContainerDied","Data":"767c04fb670b3932880fb0df8bd509adc25bdb809b40e6eba68771abd3b3f138"}
Oct 30 19:58:50.254524 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.254500    5704 scope.go:122] "RemoveContainer" containerID="e6add73921011ae7a832f5fdd6d4ed385ac721e46742ec325bb1ac09f119e13f"
Oct 30 19:58:50.254879 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:58:50.254855    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6add73921011ae7a832f5fdd6d4ed385ac721e46742ec325bb1ac09f119e13f\": container with ID starting with e6add73921011ae7a832f5fdd6d4ed385ac721e46742ec325bb1ac09f119e13f not found: ID does not exist" containerID="e6add73921011ae7a832f5fdd6d4ed385ac721e46742ec325bb1ac09f119e13f"
Oct 30 19:58:50.254941 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.254889    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6add73921011ae7a832f5fdd6d4ed385ac721e46742ec325bb1ac09f119e13f"} err="failed to get container status \"e6add73921011ae7a832f5fdd6d4ed385ac721e46742ec325bb1ac09f119e13f\": rpc error: code = NotFound desc = could not find container \"e6add73921011ae7a832f5fdd6d4ed385ac721e46742ec325bb1ac09f119e13f\": container with ID starting with e6add73921011ae7a832f5fdd6d4ed385ac721e46742ec325bb1ac09f119e13f not found: ID does not exist"
Oct 30 19:58:50.259254 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.259233    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"
Oct 30 19:58:50.333958 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.333931    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/57bc5590-a521-4cdd-9f55-7667b13bcfba-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57bc5590-a521-4cdd-9f55-7667b13bcfba-plugins-config-volume\") pod \"57bc5590-a521-4cdd-9f55-7667b13bcfba\" (UID: \"57bc5590-a521-4cdd-9f55-7667b13bcfba\") "
Oct 30 19:58:50.333958 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.333978    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/57bc5590-a521-4cdd-9f55-7667b13bcfba-kube-api-access-8m7w4\" (UniqueName: \"kubernetes.io/projected/57bc5590-a521-4cdd-9f55-7667b13bcfba-kube-api-access-8m7w4\") pod \"57bc5590-a521-4cdd-9f55-7667b13bcfba\" (UID: \"57bc5590-a521-4cdd-9f55-7667b13bcfba\") "
Oct 30 19:58:50.334381 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.334325    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57bc5590-a521-4cdd-9f55-7667b13bcfba-plugins-config-volume" pod "57bc5590-a521-4cdd-9f55-7667b13bcfba" (UID: "57bc5590-a521-4cdd-9f55-7667b13bcfba"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:50.334488 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.334424    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz6xf\" (UniqueName: \"kubernetes.io/projected/1803662c-638a-4442-b019-1ef97209f2a8-kube-api-access-hz6xf\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:50.336043 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.336014    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57bc5590-a521-4cdd-9f55-7667b13bcfba-kube-api-access-8m7w4" pod "57bc5590-a521-4cdd-9f55-7667b13bcfba" (UID: "57bc5590-a521-4cdd-9f55-7667b13bcfba"). InnerVolumeSpecName "kube-api-access-8m7w4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:50.393545 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.393521    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"]
Oct 30 19:58:50.398007 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.397987    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-22fb18-869d9bc646-nzpp7"]
Oct 30 19:58:50.435192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.435142    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57bc5590-a521-4cdd-9f55-7667b13bcfba-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:50.435192 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.435160    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m7w4\" (UniqueName: \"kubernetes.io/projected/57bc5590-a521-4cdd-9f55-7667b13bcfba-kube-api-access-8m7w4\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:50.988617 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:50.988586    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx" podUID="17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 19:58:51.178926 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:51.178855    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1803662c-638a-4442-b019-1ef97209f2a8" path="/var/lib/kubelet/pods/1803662c-638a-4442-b019-1ef97209f2a8/volumes"
Oct 30 19:58:51.248429 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:51.248403    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"
Oct 30 19:58:51.248572 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:51.248429    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx" event={"ID":"57bc5590-a521-4cdd-9f55-7667b13bcfba","Type":"ContainerDied","Data":"b7ad1d596ab03c4afd5f1b161286df79e19cad90fa9391527f19d8ceba301b81"}
Oct 30 19:58:51.248572 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:51.248477    5704 scope.go:122] "RemoveContainer" containerID="767c04fb670b3932880fb0df8bd509adc25bdb809b40e6eba68771abd3b3f138"
Oct 30 19:58:51.402764 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:51.402735    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"]
Oct 30 19:58:51.406643 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:51.406616    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-22fb18-6d9d4d4c7f-p44rx"]
Oct 30 19:58:51.561582 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:51.561549    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:51.800513 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:51.800477    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:51.979697 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:51.979609    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:52.618734 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.618679    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"
Oct 30 19:58:52.754286 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.753976    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"
Oct 30 19:58:52.761064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.760456    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20d434f7-33e5-4136-9114-43d281037737-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d434f7-33e5-4136-9114-43d281037737-plugins-config-volume\") pod \"20d434f7-33e5-4136-9114-43d281037737\" (UID: \"20d434f7-33e5-4136-9114-43d281037737\") "
Oct 30 19:58:52.761064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.760700    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20d434f7-33e5-4136-9114-43d281037737-kube-api-access-rwptf\" (UniqueName: \"kubernetes.io/projected/20d434f7-33e5-4136-9114-43d281037737-kube-api-access-rwptf\") pod \"20d434f7-33e5-4136-9114-43d281037737\" (UID: \"20d434f7-33e5-4136-9114-43d281037737\") "
Oct 30 19:58:52.761064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.760891    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20d434f7-33e5-4136-9114-43d281037737-plugins-config-volume" pod "20d434f7-33e5-4136-9114-43d281037737" (UID: "20d434f7-33e5-4136-9114-43d281037737"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:52.761064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.761011    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/20d434f7-33e5-4136-9114-43d281037737-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:52.763598 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.763384    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20d434f7-33e5-4136-9114-43d281037737-kube-api-access-rwptf" pod "20d434f7-33e5-4136-9114-43d281037737" (UID: "20d434f7-33e5-4136-9114-43d281037737"). InnerVolumeSpecName "kube-api-access-rwptf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:52.861684 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.861654    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f1c467c6-c1dc-4095-9849-35acde68f3a3-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c467c6-c1dc-4095-9849-35acde68f3a3-plugins-config-volume\") pod \"f1c467c6-c1dc-4095-9849-35acde68f3a3\" (UID: \"f1c467c6-c1dc-4095-9849-35acde68f3a3\") "
Oct 30 19:58:52.861879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.861695    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f1c467c6-c1dc-4095-9849-35acde68f3a3-kube-api-access-x5m54\" (UniqueName: \"kubernetes.io/projected/f1c467c6-c1dc-4095-9849-35acde68f3a3-kube-api-access-x5m54\") pod \"f1c467c6-c1dc-4095-9849-35acde68f3a3\" (UID: \"f1c467c6-c1dc-4095-9849-35acde68f3a3\") "
Oct 30 19:58:52.861994 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.861978    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwptf\" (UniqueName: \"kubernetes.io/projected/20d434f7-33e5-4136-9114-43d281037737-kube-api-access-rwptf\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:52.862113 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.861989    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1c467c6-c1dc-4095-9849-35acde68f3a3-plugins-config-volume" pod "f1c467c6-c1dc-4095-9849-35acde68f3a3" (UID: "f1c467c6-c1dc-4095-9849-35acde68f3a3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:52.863740 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.863723    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f1c467c6-c1dc-4095-9849-35acde68f3a3-kube-api-access-x5m54" pod "f1c467c6-c1dc-4095-9849-35acde68f3a3" (UID: "f1c467c6-c1dc-4095-9849-35acde68f3a3"). InnerVolumeSpecName "kube-api-access-x5m54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:52.962709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.962628    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f1c467c6-c1dc-4095-9849-35acde68f3a3-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:52.962709 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:52.962654    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5m54\" (UniqueName: \"kubernetes.io/projected/f1c467c6-c1dc-4095-9849-35acde68f3a3-kube-api-access-x5m54\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:53.179202 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.179172    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="57bc5590-a521-4cdd-9f55-7667b13bcfba" path="/var/lib/kubelet/pods/57bc5590-a521-4cdd-9f55-7667b13bcfba/volumes"
Oct 30 19:58:53.262233 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.262203    5704 generic.go:356] "Generic (PLEG): container finished" podID="f1c467c6-c1dc-4095-9849-35acde68f3a3" containerID="9d88f0bd77097c3a6120097a075cd90341af362201886527149c18ad8637910d" exitCode=0
Oct 30 19:58:53.262393 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.262255    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"
Oct 30 19:58:53.262393 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.262272    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62" event={"ID":"f1c467c6-c1dc-4095-9849-35acde68f3a3","Type":"ContainerDied","Data":"9d88f0bd77097c3a6120097a075cd90341af362201886527149c18ad8637910d"}
Oct 30 19:58:53.262393 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.262343    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62" event={"ID":"f1c467c6-c1dc-4095-9849-35acde68f3a3","Type":"ContainerDied","Data":"5c96b6955acb9c83606f1e284f71c6a58fb86c81c7c73fbbfe76e89924ddd4bb"}
Oct 30 19:58:53.262393 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.262374    5704 scope.go:122] "RemoveContainer" containerID="9d88f0bd77097c3a6120097a075cd90341af362201886527149c18ad8637910d"
Oct 30 19:58:53.263759 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.263735    5704 generic.go:356] "Generic (PLEG): container finished" podID="20d434f7-33e5-4136-9114-43d281037737" containerID="a21ab7d115bde8c9803c85a09ccc758064afceeff6b7caf01d2551950e1c9b8f" exitCode=0
Oct 30 19:58:53.263851 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.263795    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2" event={"ID":"20d434f7-33e5-4136-9114-43d281037737","Type":"ContainerDied","Data":"a21ab7d115bde8c9803c85a09ccc758064afceeff6b7caf01d2551950e1c9b8f"}
Oct 30 19:58:53.263851 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.263836    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2" event={"ID":"20d434f7-33e5-4136-9114-43d281037737","Type":"ContainerDied","Data":"cd7e4c63af3b39a068ac79f64943f9c6a9480ad3aac63029891ba707e4f8fd36"}
Oct 30 19:58:53.263937 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.263800    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"
Oct 30 19:58:53.271585 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.271570    5704 scope.go:122] "RemoveContainer" containerID="9d88f0bd77097c3a6120097a075cd90341af362201886527149c18ad8637910d"
Oct 30 19:58:53.271864 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:58:53.271844    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d88f0bd77097c3a6120097a075cd90341af362201886527149c18ad8637910d\": container with ID starting with 9d88f0bd77097c3a6120097a075cd90341af362201886527149c18ad8637910d not found: ID does not exist" containerID="9d88f0bd77097c3a6120097a075cd90341af362201886527149c18ad8637910d"
Oct 30 19:58:53.271970 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.271868    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d88f0bd77097c3a6120097a075cd90341af362201886527149c18ad8637910d"} err="failed to get container status \"9d88f0bd77097c3a6120097a075cd90341af362201886527149c18ad8637910d\": rpc error: code = NotFound desc = could not find container \"9d88f0bd77097c3a6120097a075cd90341af362201886527149c18ad8637910d\": container with ID starting with 9d88f0bd77097c3a6120097a075cd90341af362201886527149c18ad8637910d not found: ID does not exist"
Oct 30 19:58:53.271970 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.271883    5704 scope.go:122] "RemoveContainer" containerID="a21ab7d115bde8c9803c85a09ccc758064afceeff6b7caf01d2551950e1c9b8f"
Oct 30 19:58:53.282733 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.282713    5704 scope.go:122] "RemoveContainer" containerID="a21ab7d115bde8c9803c85a09ccc758064afceeff6b7caf01d2551950e1c9b8f"
Oct 30 19:58:53.282999 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:58:53.282981    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a21ab7d115bde8c9803c85a09ccc758064afceeff6b7caf01d2551950e1c9b8f\": container with ID starting with a21ab7d115bde8c9803c85a09ccc758064afceeff6b7caf01d2551950e1c9b8f not found: ID does not exist" containerID="a21ab7d115bde8c9803c85a09ccc758064afceeff6b7caf01d2551950e1c9b8f"
Oct 30 19:58:53.283072 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.283005    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a21ab7d115bde8c9803c85a09ccc758064afceeff6b7caf01d2551950e1c9b8f"} err="failed to get container status \"a21ab7d115bde8c9803c85a09ccc758064afceeff6b7caf01d2551950e1c9b8f\": rpc error: code = NotFound desc = could not find container \"a21ab7d115bde8c9803c85a09ccc758064afceeff6b7caf01d2551950e1c9b8f\": container with ID starting with a21ab7d115bde8c9803c85a09ccc758064afceeff6b7caf01d2551950e1c9b8f not found: ID does not exist"
Oct 30 19:58:53.422057 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.422020    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"]
Oct 30 19:58:53.467630 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.467603    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-85f471-7d68cdc4b-p8pr2"]
Oct 30 19:58:53.531536 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.531473    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"]
Oct 30 19:58:53.536308 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.536288    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-85f471-56b998fd47-2nq62"]
Oct 30 19:58:53.996924 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:53.991431    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:55.178818 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:55.178787    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="20d434f7-33e5-4136-9114-43d281037737" path="/var/lib/kubelet/pods/20d434f7-33e5-4136-9114-43d281037737/volumes"
Oct 30 19:58:55.179380 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:55.179345    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f1c467c6-c1dc-4095-9849-35acde68f3a3" path="/var/lib/kubelet/pods/f1c467c6-c1dc-4095-9849-35acde68f3a3/volumes"
Oct 30 19:58:55.619625 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:55.619546    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:55.846008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:55.845968    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:56.834879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:56.834844    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:57.209871 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.209785    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl" podUID="e84096c6-01ea-407c-8d70-46f7b14bde3f" containerName="main" containerID="cri-o://e4473e938b1d94f74460f48fea3d6f3effd526d4467694c2ea1ff693574fa985" gracePeriod=45
Oct 30 19:58:57.210083 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.210053    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" podUID="bba0064f-500b-4920-88e7-43dc15b56e3e" containerName="main" containerID="cri-o://8d5fc9de854a2dc3b211e0c34870340f79af1e78a4de75fd3e187957fed7d5d6" gracePeriod=45
Oct 30 19:58:57.231395 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.231363    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c" podUID="1d7b5569-0635-4061-b3d7-eb9890ad2184" containerName="main" containerID="cri-o://c7ffc66d6f14f509cb82bfe98b131b53ecdecc3cd891a9cbc372e8e2c8e3a5ec" gracePeriod=45
Oct 30 19:58:57.395977 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.395649    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl" podUID="e84096c6-01ea-407c-8d70-46f7b14bde3f" containerName="main" probeResult="failure" output="Get \"https://10.134.0.68:8000/ready\": dial tcp 10.134.0.68:8000: connect: connection refused"
Oct 30 19:58:57.520398 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.520375    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:58:57.524106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.524085    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:58:57.537583 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.537564    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:58:57.611291 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611265    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d7b5569-0635-4061-b3d7-eb9890ad2184-kube-api-access-fgbxg\" (UniqueName: \"kubernetes.io/projected/1d7b5569-0635-4061-b3d7-eb9890ad2184-kube-api-access-fgbxg\") pod \"1d7b5569-0635-4061-b3d7-eb9890ad2184\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") "
Oct 30 19:58:57.611451 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611307    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-home\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-home\") pod \"1d7b5569-0635-4061-b3d7-eb9890ad2184\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") "
Oct 30 19:58:57.611451 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611343    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bba0064f-500b-4920-88e7-43dc15b56e3e-kube-api-access-dqghz\" (UniqueName: \"kubernetes.io/projected/bba0064f-500b-4920-88e7-43dc15b56e3e-kube-api-access-dqghz\") pod \"bba0064f-500b-4920-88e7-43dc15b56e3e\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") "
Oct 30 19:58:57.611451 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611387    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e84096c6-01ea-407c-8d70-46f7b14bde3f-kube-api-access-lx9cl\" (UniqueName: \"kubernetes.io/projected/e84096c6-01ea-407c-8d70-46f7b14bde3f-kube-api-access-lx9cl\") pod \"e84096c6-01ea-407c-8d70-46f7b14bde3f\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") "
Oct 30 19:58:57.611451 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611428    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-dshm\") pod \"1d7b5569-0635-4061-b3d7-eb9890ad2184\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") "
Oct 30 19:58:57.611689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611458    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-tmp-dir\") pod \"1d7b5569-0635-4061-b3d7-eb9890ad2184\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") "
Oct 30 19:58:57.611689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611486    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-dshm\") pod \"e84096c6-01ea-407c-8d70-46f7b14bde3f\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") "
Oct 30 19:58:57.611689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611528    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-model-cache\") pod \"bba0064f-500b-4920-88e7-43dc15b56e3e\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") "
Oct 30 19:58:57.611689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611576    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e84096c6-01ea-407c-8d70-46f7b14bde3f-tls-certs\" (UniqueName: \"kubernetes.io/secret/e84096c6-01ea-407c-8d70-46f7b14bde3f-tls-certs\") pod \"e84096c6-01ea-407c-8d70-46f7b14bde3f\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") "
Oct 30 19:58:57.611689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611591    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-home" pod "1d7b5569-0635-4061-b3d7-eb9890ad2184" (UID: "1d7b5569-0635-4061-b3d7-eb9890ad2184"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.611689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611616    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/1d7b5569-0635-4061-b3d7-eb9890ad2184-tls-certs\" (UniqueName: \"kubernetes.io/secret/1d7b5569-0635-4061-b3d7-eb9890ad2184-tls-certs\") pod \"1d7b5569-0635-4061-b3d7-eb9890ad2184\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") "
Oct 30 19:58:57.611689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611667    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bba0064f-500b-4920-88e7-43dc15b56e3e-tls-certs\" (UniqueName: \"kubernetes.io/secret/bba0064f-500b-4920-88e7-43dc15b56e3e-tls-certs\") pod \"bba0064f-500b-4920-88e7-43dc15b56e3e\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") "
Oct 30 19:58:57.612095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611694    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-home\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-home\") pod \"e84096c6-01ea-407c-8d70-46f7b14bde3f\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") "
Oct 30 19:58:57.612095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611729    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-dshm\") pod \"bba0064f-500b-4920-88e7-43dc15b56e3e\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") "
Oct 30 19:58:57.612095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611757    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-tmp-dir\") pod \"e84096c6-01ea-407c-8d70-46f7b14bde3f\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") "
Oct 30 19:58:57.612095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611780    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-model-cache" pod "bba0064f-500b-4920-88e7-43dc15b56e3e" (UID: "bba0064f-500b-4920-88e7-43dc15b56e3e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.612095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611787    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-model-cache\") pod \"e84096c6-01ea-407c-8d70-46f7b14bde3f\" (UID: \"e84096c6-01ea-407c-8d70-46f7b14bde3f\") "
Oct 30 19:58:57.612095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611867    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-model-cache\") pod \"1d7b5569-0635-4061-b3d7-eb9890ad2184\" (UID: \"1d7b5569-0635-4061-b3d7-eb9890ad2184\") "
Oct 30 19:58:57.612095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611922    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-tmp-dir\") pod \"bba0064f-500b-4920-88e7-43dc15b56e3e\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") "
Oct 30 19:58:57.612095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611951    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-home" pod "e84096c6-01ea-407c-8d70-46f7b14bde3f" (UID: "e84096c6-01ea-407c-8d70-46f7b14bde3f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.612095 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.611964    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-home\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-home\") pod \"bba0064f-500b-4920-88e7-43dc15b56e3e\" (UID: \"bba0064f-500b-4920-88e7-43dc15b56e3e\") "
Oct 30 19:58:57.612662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.612494    5704 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-home\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.612662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.612517    5704 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-model-cache\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.612662 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.612532    5704 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-home\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.612828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.612706    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-home" pod "bba0064f-500b-4920-88e7-43dc15b56e3e" (UID: "bba0064f-500b-4920-88e7-43dc15b56e3e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.612828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.612746    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-tmp-dir" pod "1d7b5569-0635-4061-b3d7-eb9890ad2184" (UID: "1d7b5569-0635-4061-b3d7-eb9890ad2184"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.613403 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.613375    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-model-cache" pod "e84096c6-01ea-407c-8d70-46f7b14bde3f" (UID: "e84096c6-01ea-407c-8d70-46f7b14bde3f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.613586 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.613560    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-model-cache" pod "1d7b5569-0635-4061-b3d7-eb9890ad2184" (UID: "1d7b5569-0635-4061-b3d7-eb9890ad2184"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.613872 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.613847    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-tmp-dir" pod "e84096c6-01ea-407c-8d70-46f7b14bde3f" (UID: "e84096c6-01ea-407c-8d70-46f7b14bde3f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.614275 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.614247    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bba0064f-500b-4920-88e7-43dc15b56e3e-kube-api-access-dqghz" pod "bba0064f-500b-4920-88e7-43dc15b56e3e" (UID: "bba0064f-500b-4920-88e7-43dc15b56e3e"). InnerVolumeSpecName "kube-api-access-dqghz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:57.614388 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.614361    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-tmp-dir" pod "bba0064f-500b-4920-88e7-43dc15b56e3e" (UID: "bba0064f-500b-4920-88e7-43dc15b56e3e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.614388 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.614385    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e84096c6-01ea-407c-8d70-46f7b14bde3f-tls-certs" pod "e84096c6-01ea-407c-8d70-46f7b14bde3f" (UID: "e84096c6-01ea-407c-8d70-46f7b14bde3f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:57.614689 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.614661    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d7b5569-0635-4061-b3d7-eb9890ad2184-kube-api-access-fgbxg" pod "1d7b5569-0635-4061-b3d7-eb9890ad2184" (UID: "1d7b5569-0635-4061-b3d7-eb9890ad2184"). InnerVolumeSpecName "kube-api-access-fgbxg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:57.615410 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.615374    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-dshm" pod "1d7b5569-0635-4061-b3d7-eb9890ad2184" (UID: "1d7b5569-0635-4061-b3d7-eb9890ad2184"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.615546 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.615522    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-dshm" pod "e84096c6-01ea-407c-8d70-46f7b14bde3f" (UID: "e84096c6-01ea-407c-8d70-46f7b14bde3f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.616455 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.616429    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d7b5569-0635-4061-b3d7-eb9890ad2184-tls-certs" pod "1d7b5569-0635-4061-b3d7-eb9890ad2184" (UID: "1d7b5569-0635-4061-b3d7-eb9890ad2184"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:57.616611 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.616584    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bba0064f-500b-4920-88e7-43dc15b56e3e-tls-certs" pod "bba0064f-500b-4920-88e7-43dc15b56e3e" (UID: "bba0064f-500b-4920-88e7-43dc15b56e3e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:58:57.616700 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.616611    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e84096c6-01ea-407c-8d70-46f7b14bde3f-kube-api-access-lx9cl" pod "e84096c6-01ea-407c-8d70-46f7b14bde3f" (UID: "e84096c6-01ea-407c-8d70-46f7b14bde3f"). InnerVolumeSpecName "kube-api-access-lx9cl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:57.617081 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.617022    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-dshm" pod "bba0064f-500b-4920-88e7-43dc15b56e3e" (UID: "bba0064f-500b-4920-88e7-43dc15b56e3e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:58:57.713775 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713747    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgbxg\" (UniqueName: \"kubernetes.io/projected/1d7b5569-0635-4061-b3d7-eb9890ad2184-kube-api-access-fgbxg\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713775 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713774    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqghz\" (UniqueName: \"kubernetes.io/projected/bba0064f-500b-4920-88e7-43dc15b56e3e-kube-api-access-dqghz\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713784    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx9cl\" (UniqueName: \"kubernetes.io/projected/e84096c6-01ea-407c-8d70-46f7b14bde3f-kube-api-access-lx9cl\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713794    5704 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-dshm\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713803    5704 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-tmp-dir\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713814    5704 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-dshm\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713824    5704 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e84096c6-01ea-407c-8d70-46f7b14bde3f-tls-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713831    5704 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d7b5569-0635-4061-b3d7-eb9890ad2184-tls-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713840    5704 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bba0064f-500b-4920-88e7-43dc15b56e3e-tls-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713848    5704 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-dshm\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713856    5704 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-tmp-dir\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713863    5704 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e84096c6-01ea-407c-8d70-46f7b14bde3f-model-cache\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713872    5704 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d7b5569-0635-4061-b3d7-eb9890ad2184-model-cache\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713881    5704 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-tmp-dir\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:57.713967 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:57.713893    5704 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bba0064f-500b-4920-88e7-43dc15b56e3e-home\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:58.181051 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.181014    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"
Oct 30 19:58:58.294433 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.294405    5704 generic.go:356] "Generic (PLEG): container finished" podID="1d7b5569-0635-4061-b3d7-eb9890ad2184" containerID="c7ffc66d6f14f509cb82bfe98b131b53ecdecc3cd891a9cbc372e8e2c8e3a5ec" exitCode=0
Oct 30 19:58:58.294621 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.294443    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"
Oct 30 19:58:58.294621 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.294488    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c" event={"ID":"1d7b5569-0635-4061-b3d7-eb9890ad2184","Type":"ContainerDied","Data":"c7ffc66d6f14f509cb82bfe98b131b53ecdecc3cd891a9cbc372e8e2c8e3a5ec"}
Oct 30 19:58:58.294621 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.294548    5704 scope.go:122] "RemoveContainer" containerID="c7ffc66d6f14f509cb82bfe98b131b53ecdecc3cd891a9cbc372e8e2c8e3a5ec"
Oct 30 19:58:58.295588 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.294538    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c" event={"ID":"1d7b5569-0635-4061-b3d7-eb9890ad2184","Type":"ContainerDied","Data":"5b04c41c7b1ebda5bfb40d1e5c776b99ebb13fc506d20dded1c8c7f505fa7a8c"}
Oct 30 19:58:58.306667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.305902    5704 generic.go:356] "Generic (PLEG): container finished" podID="bba0064f-500b-4920-88e7-43dc15b56e3e" containerID="8d5fc9de854a2dc3b211e0c34870340f79af1e78a4de75fd3e187957fed7d5d6" exitCode=0
Oct 30 19:58:58.306667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.306042    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" event={"ID":"bba0064f-500b-4920-88e7-43dc15b56e3e","Type":"ContainerDied","Data":"8d5fc9de854a2dc3b211e0c34870340f79af1e78a4de75fd3e187957fed7d5d6"}
Oct 30 19:58:58.306667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.306103    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" event={"ID":"bba0064f-500b-4920-88e7-43dc15b56e3e","Type":"ContainerDied","Data":"60ef52e11003631a87183af6a0cbc0aea2a3086de4bb8f48d228ceb6aee0641c"}
Oct 30 19:58:58.306667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.306207    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"
Oct 30 19:58:58.309862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.309421    5704 generic.go:356] "Generic (PLEG): container finished" podID="17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b" containerID="8dd731f7e3035379faac95addd792db5037833e98f177fa21d9abec395fd9800" exitCode=0
Oct 30 19:58:58.309862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.309669    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx" event={"ID":"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b","Type":"ContainerDied","Data":"8dd731f7e3035379faac95addd792db5037833e98f177fa21d9abec395fd9800"}
Oct 30 19:58:58.309862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.309737    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"
Oct 30 19:58:58.309862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.309746    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx" event={"ID":"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b","Type":"ContainerDied","Data":"00db81ab8dd93d8dc136658e0e0428e5c4936dd9744bdf764038d877ef778490"}
Oct 30 19:58:58.312006 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.311983    5704 generic.go:356] "Generic (PLEG): container finished" podID="e84096c6-01ea-407c-8d70-46f7b14bde3f" containerID="e4473e938b1d94f74460f48fea3d6f3effd526d4467694c2ea1ff693574fa985" exitCode=0
Oct 30 19:58:58.312132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.312068    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl" event={"ID":"e84096c6-01ea-407c-8d70-46f7b14bde3f","Type":"ContainerDied","Data":"e4473e938b1d94f74460f48fea3d6f3effd526d4467694c2ea1ff693574fa985"}
Oct 30 19:58:58.312132 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.312099    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl" event={"ID":"e84096c6-01ea-407c-8d70-46f7b14bde3f","Type":"ContainerDied","Data":"dcec367c545932d636d385c7d60f49bef05c0b6087d23d662245a68275c1fa03"}
Oct 30 19:58:58.312273 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.312189    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"
Oct 30 19:58:58.319591 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.319570    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-kube-api-access-95bhl\" (UniqueName: \"kubernetes.io/projected/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-kube-api-access-95bhl\") pod \"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b\" (UID: \"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b\") "
Oct 30 19:58:58.319682 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.319613    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-plugins-config-volume\") pod \"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b\" (UID: \"17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b\") "
Oct 30 19:58:58.319994 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.319974    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-plugins-config-volume" pod "17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b" (UID: "17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:58:58.322189 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.322174    5704 scope.go:122] "RemoveContainer" containerID="c7ffc66d6f14f509cb82bfe98b131b53ecdecc3cd891a9cbc372e8e2c8e3a5ec"
Oct 30 19:58:58.322256 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.322189    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-kube-api-access-95bhl" pod "17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b" (UID: "17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b"). InnerVolumeSpecName "kube-api-access-95bhl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:58:58.322464 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:58:58.322444    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7ffc66d6f14f509cb82bfe98b131b53ecdecc3cd891a9cbc372e8e2c8e3a5ec\": container with ID starting with c7ffc66d6f14f509cb82bfe98b131b53ecdecc3cd891a9cbc372e8e2c8e3a5ec not found: ID does not exist" containerID="c7ffc66d6f14f509cb82bfe98b131b53ecdecc3cd891a9cbc372e8e2c8e3a5ec"
Oct 30 19:58:58.322545 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.322470    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7ffc66d6f14f509cb82bfe98b131b53ecdecc3cd891a9cbc372e8e2c8e3a5ec"} err="failed to get container status \"c7ffc66d6f14f509cb82bfe98b131b53ecdecc3cd891a9cbc372e8e2c8e3a5ec\": rpc error: code = NotFound desc = could not find container \"c7ffc66d6f14f509cb82bfe98b131b53ecdecc3cd891a9cbc372e8e2c8e3a5ec\": container with ID starting with c7ffc66d6f14f509cb82bfe98b131b53ecdecc3cd891a9cbc372e8e2c8e3a5ec not found: ID does not exist"
Oct 30 19:58:58.322545 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.322487    5704 scope.go:122] "RemoveContainer" containerID="8d5fc9de854a2dc3b211e0c34870340f79af1e78a4de75fd3e187957fed7d5d6"
Oct 30 19:58:58.337809 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.337792    5704 scope.go:122] "RemoveContainer" containerID="8d5fc9de854a2dc3b211e0c34870340f79af1e78a4de75fd3e187957fed7d5d6"
Oct 30 19:58:58.338120 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:58:58.338102    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d5fc9de854a2dc3b211e0c34870340f79af1e78a4de75fd3e187957fed7d5d6\": container with ID starting with 8d5fc9de854a2dc3b211e0c34870340f79af1e78a4de75fd3e187957fed7d5d6 not found: ID does not exist" containerID="8d5fc9de854a2dc3b211e0c34870340f79af1e78a4de75fd3e187957fed7d5d6"
Oct 30 19:58:58.338255 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.338128    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d5fc9de854a2dc3b211e0c34870340f79af1e78a4de75fd3e187957fed7d5d6"} err="failed to get container status \"8d5fc9de854a2dc3b211e0c34870340f79af1e78a4de75fd3e187957fed7d5d6\": rpc error: code = NotFound desc = could not find container \"8d5fc9de854a2dc3b211e0c34870340f79af1e78a4de75fd3e187957fed7d5d6\": container with ID starting with 8d5fc9de854a2dc3b211e0c34870340f79af1e78a4de75fd3e187957fed7d5d6 not found: ID does not exist"
Oct 30 19:58:58.338255 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.338150    5704 scope.go:122] "RemoveContainer" containerID="8dd731f7e3035379faac95addd792db5037833e98f177fa21d9abec395fd9800"
Oct 30 19:58:58.346129 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.346109    5704 scope.go:122] "RemoveContainer" containerID="8dd731f7e3035379faac95addd792db5037833e98f177fa21d9abec395fd9800"
Oct 30 19:58:58.346361 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:58:58.346344    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dd731f7e3035379faac95addd792db5037833e98f177fa21d9abec395fd9800\": container with ID starting with 8dd731f7e3035379faac95addd792db5037833e98f177fa21d9abec395fd9800 not found: ID does not exist" containerID="8dd731f7e3035379faac95addd792db5037833e98f177fa21d9abec395fd9800"
Oct 30 19:58:58.346410 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.346366    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dd731f7e3035379faac95addd792db5037833e98f177fa21d9abec395fd9800"} err="failed to get container status \"8dd731f7e3035379faac95addd792db5037833e98f177fa21d9abec395fd9800\": rpc error: code = NotFound desc = could not find container \"8dd731f7e3035379faac95addd792db5037833e98f177fa21d9abec395fd9800\": container with ID starting with 8dd731f7e3035379faac95addd792db5037833e98f177fa21d9abec395fd9800 not found: ID does not exist"
Oct 30 19:58:58.346410 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.346380    5704 scope.go:122] "RemoveContainer" containerID="e4473e938b1d94f74460f48fea3d6f3effd526d4467694c2ea1ff693574fa985"
Oct 30 19:58:58.353936 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.353919    5704 scope.go:122] "RemoveContainer" containerID="e4473e938b1d94f74460f48fea3d6f3effd526d4467694c2ea1ff693574fa985"
Oct 30 19:58:58.354183 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:58:58.354166    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4473e938b1d94f74460f48fea3d6f3effd526d4467694c2ea1ff693574fa985\": container with ID starting with e4473e938b1d94f74460f48fea3d6f3effd526d4467694c2ea1ff693574fa985 not found: ID does not exist" containerID="e4473e938b1d94f74460f48fea3d6f3effd526d4467694c2ea1ff693574fa985"
Oct 30 19:58:58.354253 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.354187    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4473e938b1d94f74460f48fea3d6f3effd526d4467694c2ea1ff693574fa985"} err="failed to get container status \"e4473e938b1d94f74460f48fea3d6f3effd526d4467694c2ea1ff693574fa985\": rpc error: code = NotFound desc = could not find container \"e4473e938b1d94f74460f48fea3d6f3effd526d4467694c2ea1ff693574fa985\": container with ID starting with e4473e938b1d94f74460f48fea3d6f3effd526d4467694c2ea1ff693574fa985 not found: ID does not exist"
Oct 30 19:58:58.421363 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.421335    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95bhl\" (UniqueName: \"kubernetes.io/projected/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-kube-api-access-95bhl\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:58.421363 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.421360    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:58:58.453856 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.453834    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"]
Oct 30 19:58:58.458292 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.458273    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb"]
Oct 30 19:58:58.483136 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.483077    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-2301cb/distinct-w0-2301cb-kserve-7cf7f446d4-zn9rb" podUID="bba0064f-500b-4920-88e7-43dc15b56e3e" containerName="main" probeResult="failure" output="Get \"https://10.134.0.63:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Oct 30 19:58:58.565521 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.565495    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"]
Oct 30 19:58:58.570017 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.569996    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/distinct-2-w0-2301cb-kserve-5676d69f4c-t4ksl"]
Oct 30 19:58:58.722901 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.722877    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"]
Oct 30 19:58:58.728112 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.728092    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-2301cb/simulator-w0-2301cb-kserve-577dff5f4d-7h56c"]
Oct 30 19:58:58.830714 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.830684    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"]
Oct 30 19:58:58.834995 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.834968    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w1-9275c3-5f987959bc-2vtdx"]
Oct 30 19:58:58.995859 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:58.995809    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:58:59.178847 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:59.178766    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b" path="/var/lib/kubelet/pods/17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b/volumes"
Oct 30 19:58:59.179225 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:59.179210    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d7b5569-0635-4061-b3d7-eb9890ad2184" path="/var/lib/kubelet/pods/1d7b5569-0635-4061-b3d7-eb9890ad2184/volumes"
Oct 30 19:58:59.179590 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:59.179578    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bba0064f-500b-4920-88e7-43dc15b56e3e" path="/var/lib/kubelet/pods/bba0064f-500b-4920-88e7-43dc15b56e3e/volumes"
Oct 30 19:58:59.180258 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:58:59.180243    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e84096c6-01ea-407c-8d70-46f7b14bde3f" path="/var/lib/kubelet/pods/e84096c6-01ea-407c-8d70-46f7b14bde3f/volumes"
Oct 30 19:59:02.414064 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:02.414017    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:02.644241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:02.644204    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:03.150851 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:03.150822    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:03.384220 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:59:03.384174    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:59:04.540293 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:04.540263    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:59:04.660937 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:04.660907    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:04.933851 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:04.933764    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:05.148863 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:05.148822    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:05.571631 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:05.571593    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:05.761351 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:05.761314    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:06.872266 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:06.872232    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:10.202529 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:10.202489    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:13.002464 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:13.002426    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:13.210749 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:13.210715    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:13.347202 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:13.347131    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"]
Oct 30 19:59:13.347354 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:13.347338    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerName="istio-proxy" containerID="cri-o://aced46024a8e2981a595d19d08043040ac3b9eab0611779ca4dc98a53536bdb2" gracePeriod=30
Oct 30 19:59:13.406815 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:13.406787    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:13.664989 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:13.664900    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:15.013301 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:15.013263    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:18.609230 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.609202    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:59:18.639590 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.639533    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-podinfo\") pod \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") "
Oct 30 19:59:18.639719 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.639599    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-token\" (UniqueName: \"kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-token\") pod \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") "
Oct 30 19:59:18.639719 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.639667    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-socket\") pod \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") "
Oct 30 19:59:18.639843 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.639727    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-kube-api-access-6t49v\" (UniqueName: \"kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-kube-api-access-6t49v\") pod \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") "
Oct 30 19:59:18.639843 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.639780    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istiod-ca-cert\") pod \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") "
Oct 30 19:59:18.639843 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.639829    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-certs\") pod \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") "
Oct 30 19:59:18.640012 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.639886    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-data\") pod \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") "
Oct 30 19:59:18.640012 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.639928    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-envoy\") pod \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") "
Oct 30 19:59:18.640012 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.639951    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-socket" pod "139cc6a4-643f-41ee-a8c2-eaf2177f4052" (UID: "139cc6a4-643f-41ee-a8c2-eaf2177f4052"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:18.640012 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.639995    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-credential-socket\") pod \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\" (UID: \"139cc6a4-643f-41ee-a8c2-eaf2177f4052\") "
Oct 30 19:59:18.640247 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.640173    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-data" pod "139cc6a4-643f-41ee-a8c2-eaf2177f4052" (UID: "139cc6a4-643f-41ee-a8c2-eaf2177f4052"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:18.640247 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.640186    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istiod-ca-cert" pod "139cc6a4-643f-41ee-a8c2-eaf2177f4052" (UID: "139cc6a4-643f-41ee-a8c2-eaf2177f4052"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:18.640351 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.640335    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-credential-socket" pod "139cc6a4-643f-41ee-a8c2-eaf2177f4052" (UID: "139cc6a4-643f-41ee-a8c2-eaf2177f4052"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:18.640409 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.640327    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-certs" pod "139cc6a4-643f-41ee-a8c2-eaf2177f4052" (UID: "139cc6a4-643f-41ee-a8c2-eaf2177f4052"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:18.640600 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.640572    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:18.640717 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.640605    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:18.640717 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.640622    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:18.640717 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.640636    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:18.640717 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.640650    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:18.641879 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.641856    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-token" pod "139cc6a4-643f-41ee-a8c2-eaf2177f4052" (UID: "139cc6a4-643f-41ee-a8c2-eaf2177f4052"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:18.642011 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.641986    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-kube-api-access-6t49v" pod "139cc6a4-643f-41ee-a8c2-eaf2177f4052" (UID: "139cc6a4-643f-41ee-a8c2-eaf2177f4052"). InnerVolumeSpecName "kube-api-access-6t49v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:18.642123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.642072    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-podinfo" pod "139cc6a4-643f-41ee-a8c2-eaf2177f4052" (UID: "139cc6a4-643f-41ee-a8c2-eaf2177f4052"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:59:18.642123 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.642074    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-envoy" pod "139cc6a4-643f-41ee-a8c2-eaf2177f4052" (UID: "139cc6a4-643f-41ee-a8c2-eaf2177f4052"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:18.683952 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.683923    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:18.741433 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.741411    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:18.741433 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.741431    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:18.741562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.741440    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6t49v\" (UniqueName: \"kubernetes.io/projected/139cc6a4-643f-41ee-a8c2-eaf2177f4052-kube-api-access-6t49v\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:18.741562 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:18.741450    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/139cc6a4-643f-41ee-a8c2-eaf2177f4052-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:19.433432 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:19.433398    5704 generic.go:356] "Generic (PLEG): container finished" podID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerID="aced46024a8e2981a595d19d08043040ac3b9eab0611779ca4dc98a53536bdb2" exitCode=0
Oct 30 19:59:19.433580 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:19.433450    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"
Oct 30 19:59:19.433580 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:19.433463    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" event={"ID":"139cc6a4-643f-41ee-a8c2-eaf2177f4052","Type":"ContainerDied","Data":"aced46024a8e2981a595d19d08043040ac3b9eab0611779ca4dc98a53536bdb2"}
Oct 30 19:59:19.433580 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:19.433515    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2" event={"ID":"139cc6a4-643f-41ee-a8c2-eaf2177f4052","Type":"ContainerDied","Data":"d1657696da89222dcfbc737ab91fdb165d94729eff3d4714c82bfc23c5d647d5"}
Oct 30 19:59:19.433580 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:19.433545    5704 scope.go:122] "RemoveContainer" containerID="aced46024a8e2981a595d19d08043040ac3b9eab0611779ca4dc98a53536bdb2"
Oct 30 19:59:19.446818 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:19.446763    5704 scope.go:122] "RemoveContainer" containerID="47071d4e5aea22e8e246f9949e8093c5068faf1195413c78c5d5cd85fa934b5e"
Oct 30 19:59:19.455301 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:19.455284    5704 scope.go:122] "RemoveContainer" containerID="aced46024a8e2981a595d19d08043040ac3b9eab0611779ca4dc98a53536bdb2"
Oct 30 19:59:19.455595 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:59:19.455574    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aced46024a8e2981a595d19d08043040ac3b9eab0611779ca4dc98a53536bdb2\": container with ID starting with aced46024a8e2981a595d19d08043040ac3b9eab0611779ca4dc98a53536bdb2 not found: ID does not exist" containerID="aced46024a8e2981a595d19d08043040ac3b9eab0611779ca4dc98a53536bdb2"
Oct 30 19:59:19.455666 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:19.455611    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aced46024a8e2981a595d19d08043040ac3b9eab0611779ca4dc98a53536bdb2"} err="failed to get container status \"aced46024a8e2981a595d19d08043040ac3b9eab0611779ca4dc98a53536bdb2\": rpc error: code = NotFound desc = could not find container \"aced46024a8e2981a595d19d08043040ac3b9eab0611779ca4dc98a53536bdb2\": container with ID starting with aced46024a8e2981a595d19d08043040ac3b9eab0611779ca4dc98a53536bdb2 not found: ID does not exist"
Oct 30 19:59:19.455666 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:19.455630    5704 scope.go:122] "RemoveContainer" containerID="47071d4e5aea22e8e246f9949e8093c5068faf1195413c78c5d5cd85fa934b5e"
Oct 30 19:59:19.455873 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:59:19.455855    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47071d4e5aea22e8e246f9949e8093c5068faf1195413c78c5d5cd85fa934b5e\": container with ID starting with 47071d4e5aea22e8e246f9949e8093c5068faf1195413c78c5d5cd85fa934b5e not found: ID does not exist" containerID="47071d4e5aea22e8e246f9949e8093c5068faf1195413c78c5d5cd85fa934b5e"
Oct 30 19:59:19.455976 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:19.455884    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47071d4e5aea22e8e246f9949e8093c5068faf1195413c78c5d5cd85fa934b5e"} err="failed to get container status \"47071d4e5aea22e8e246f9949e8093c5068faf1195413c78c5d5cd85fa934b5e\": rpc error: code = NotFound desc = could not find container \"47071d4e5aea22e8e246f9949e8093c5068faf1195413c78c5d5cd85fa934b5e\": container with ID starting with 47071d4e5aea22e8e246f9949e8093c5068faf1195413c78c5d5cd85fa934b5e not found: ID does not exist"
Oct 30 19:59:19.645973 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:19.645943    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"]
Oct 30 19:59:19.691893 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:19.691863    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-85f471-openshift-default-c48b4885c-smvp2"]
Oct 30 19:59:19.930904 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:19.930873    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:20.136533 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:20.136493    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:21.178858 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:21.178829    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" path="/var/lib/kubelet/pods/139cc6a4-643f-41ee-a8c2-eaf2177f4052/volumes"
Oct 30 19:59:21.891409 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:21.891374    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:24.087454 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.087420    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"]
Oct 30 19:59:24.087903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.087645    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc" podUID="768c017a-29de-4fdc-8782-6924c6740b21" containerName="maas-api" containerID="cri-o://5fbc336b6494b53dd89d0d71dd19c5c68a8fdc8f23b226596795cfb7dc90b70f" gracePeriod=30
Oct 30 19:59:24.253442 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.253408    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:24.344952 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.344898    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:59:24.402365 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.402339    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/768c017a-29de-4fdc-8782-6924c6740b21-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768c017a-29de-4fdc-8782-6924c6740b21-openshift-service-ca\") pod \"768c017a-29de-4fdc-8782-6924c6740b21\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") "
Oct 30 19:59:24.402499 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.402431    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-maas-api-tls\") pod \"768c017a-29de-4fdc-8782-6924c6740b21\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") "
Oct 30 19:59:24.402499 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.402477    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/768c017a-29de-4fdc-8782-6924c6740b21-kube-api-access-ljq29\" (UniqueName: \"kubernetes.io/projected/768c017a-29de-4fdc-8782-6924c6740b21-kube-api-access-ljq29\") pod \"768c017a-29de-4fdc-8782-6924c6740b21\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") "
Oct 30 19:59:24.402624 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.402536    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-metrics-certs\" (UniqueName: \"kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-metrics-certs\") pod \"768c017a-29de-4fdc-8782-6924c6740b21\" (UID: \"768c017a-29de-4fdc-8782-6924c6740b21\") "
Oct 30 19:59:24.402754 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.402723    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/768c017a-29de-4fdc-8782-6924c6740b21-openshift-service-ca" pod "768c017a-29de-4fdc-8782-6924c6740b21" (UID: "768c017a-29de-4fdc-8782-6924c6740b21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:24.404593 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.404569    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/768c017a-29de-4fdc-8782-6924c6740b21-kube-api-access-ljq29" pod "768c017a-29de-4fdc-8782-6924c6740b21" (UID: "768c017a-29de-4fdc-8782-6924c6740b21"). InnerVolumeSpecName "kube-api-access-ljq29". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:24.404677 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.404600    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-metrics-certs" pod "768c017a-29de-4fdc-8782-6924c6740b21" (UID: "768c017a-29de-4fdc-8782-6924c6740b21"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:24.404677 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.404619    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-maas-api-tls" pod "768c017a-29de-4fdc-8782-6924c6740b21" (UID: "768c017a-29de-4fdc-8782-6924c6740b21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:24.470233 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.470178    5704 generic.go:356] "Generic (PLEG): container finished" podID="768c017a-29de-4fdc-8782-6924c6740b21" containerID="5fbc336b6494b53dd89d0d71dd19c5c68a8fdc8f23b226596795cfb7dc90b70f" exitCode=0
Oct 30 19:59:24.470455 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.470374    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc" event={"ID":"768c017a-29de-4fdc-8782-6924c6740b21","Type":"ContainerDied","Data":"5fbc336b6494b53dd89d0d71dd19c5c68a8fdc8f23b226596795cfb7dc90b70f"}
Oct 30 19:59:24.470455 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.470428    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc" event={"ID":"768c017a-29de-4fdc-8782-6924c6740b21","Type":"ContainerDied","Data":"844b813667c07b935e91ed3edc38e85ae9569c1cd5f4acb51f3fea7a36e9fc76"}
Oct 30 19:59:24.470455 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.470456    5704 scope.go:122] "RemoveContainer" containerID="5fbc336b6494b53dd89d0d71dd19c5c68a8fdc8f23b226596795cfb7dc90b70f"
Oct 30 19:59:24.470735 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.470607    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"
Oct 30 19:59:24.488720 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.488694    5704 scope.go:122] "RemoveContainer" containerID="5fbc336b6494b53dd89d0d71dd19c5c68a8fdc8f23b226596795cfb7dc90b70f"
Oct 30 19:59:24.489055 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:59:24.489020    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5fbc336b6494b53dd89d0d71dd19c5c68a8fdc8f23b226596795cfb7dc90b70f\": container with ID starting with 5fbc336b6494b53dd89d0d71dd19c5c68a8fdc8f23b226596795cfb7dc90b70f not found: ID does not exist" containerID="5fbc336b6494b53dd89d0d71dd19c5c68a8fdc8f23b226596795cfb7dc90b70f"
Oct 30 19:59:24.489159 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.489065    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5fbc336b6494b53dd89d0d71dd19c5c68a8fdc8f23b226596795cfb7dc90b70f"} err="failed to get container status \"5fbc336b6494b53dd89d0d71dd19c5c68a8fdc8f23b226596795cfb7dc90b70f\": rpc error: code = NotFound desc = could not find container \"5fbc336b6494b53dd89d0d71dd19c5c68a8fdc8f23b226596795cfb7dc90b70f\": container with ID starting with 5fbc336b6494b53dd89d0d71dd19c5c68a8fdc8f23b226596795cfb7dc90b70f not found: ID does not exist"
Oct 30 19:59:24.504121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.504098    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/768c017a-29de-4fdc-8782-6924c6740b21-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:24.504121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.504122    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:24.504121 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.504131    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ljq29\" (UniqueName: \"kubernetes.io/projected/768c017a-29de-4fdc-8782-6924c6740b21-kube-api-access-ljq29\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:24.504289 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.504141    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/768c017a-29de-4fdc-8782-6924c6740b21-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:24.652019 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.651935    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"]
Oct 30 19:59:24.654332 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:24.654307    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w3-22fb18-7bb97dc6dd-njssc"]
Oct 30 19:59:25.182606 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:25.182571    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="768c017a-29de-4fdc-8782-6924c6740b21" path="/var/lib/kubelet/pods/768c017a-29de-4fdc-8782-6924c6740b21/volumes"
Oct 30 19:59:26.120300 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:26.117116    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:26.343865 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:26.343826    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:26.533753 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:26.533714    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:27.062573 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:27.062532    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:27.804285 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:27.804244    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:28.000395 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:28.000364    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:28.186994 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:28.186906    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:28.412793 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:28.412752    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:28.596836 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:28.596798    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:29.189257 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:29.189189    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:29.374226 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:29.374191    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:29.550770 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:29.550737    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:29.736508 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:29.736472    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:30.344759 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:30.344727    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:31.321399 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:31.321112    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:32.387849 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.387816    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:32.896580 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.896546    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"]
Oct 30 19:59:32.897019 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897006    5704 cpu_manager.go:414] "containerMap: removing container" podUID="e84096c6-01ea-407c-8d70-46f7b14bde3f" containerName="main"
Oct 30 19:59:32.897116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897020    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e84096c6-01ea-407c-8d70-46f7b14bde3f" containerName="main"
Oct 30 19:59:32.897116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897060    5704 cpu_manager.go:414] "containerMap: removing container" podUID="902a14d3-7e9d-448c-a951-465010109b2c" containerName="istio-proxy"
Oct 30 19:59:32.897116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897066    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="902a14d3-7e9d-448c-a951-465010109b2c" containerName="istio-proxy"
Oct 30 19:59:32.897116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897077    5704 cpu_manager.go:414] "containerMap: removing container" podUID="0f628790-151a-4760-9dfd-62f0ca9c87e4" containerName="payload-processing"
Oct 30 19:59:32.897116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897082    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0f628790-151a-4760-9dfd-62f0ca9c87e4" containerName="payload-processing"
Oct 30 19:59:32.897116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897093    5704 cpu_manager.go:414] "containerMap: removing container" podUID="57bc5590-a521-4cdd-9f55-7667b13bcfba" containerName="payload-pre-processing"
Oct 30 19:59:32.897116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897097    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="57bc5590-a521-4cdd-9f55-7667b13bcfba" containerName="payload-pre-processing"
Oct 30 19:59:32.897116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897114    5704 cpu_manager.go:414] "containerMap: removing container" podUID="bba0064f-500b-4920-88e7-43dc15b56e3e" containerName="main"
Oct 30 19:59:32.897116 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897119    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bba0064f-500b-4920-88e7-43dc15b56e3e" containerName="main"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897127    5704 cpu_manager.go:414] "containerMap: removing container" podUID="768c017a-29de-4fdc-8782-6924c6740b21" containerName="maas-api"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897132    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="768c017a-29de-4fdc-8782-6924c6740b21" containerName="maas-api"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897139    5704 cpu_manager.go:414] "containerMap: removing container" podUID="17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b" containerName="payload-pre-processing"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897143    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b" containerName="payload-pre-processing"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897151    5704 cpu_manager.go:414] "containerMap: removing container" podUID="20d434f7-33e5-4136-9114-43d281037737" containerName="payload-processing"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897155    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20d434f7-33e5-4136-9114-43d281037737" containerName="payload-processing"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897171    5704 cpu_manager.go:414] "containerMap: removing container" podUID="1d7b5569-0635-4061-b3d7-eb9890ad2184" containerName="main"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897175    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d7b5569-0635-4061-b3d7-eb9890ad2184" containerName="main"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897182    5704 cpu_manager.go:414] "containerMap: removing container" podUID="1803662c-638a-4442-b019-1ef97209f2a8" containerName="payload-processing"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897186    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1803662c-638a-4442-b019-1ef97209f2a8" containerName="payload-processing"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897192    5704 cpu_manager.go:414] "containerMap: removing container" podUID="f1c467c6-c1dc-4095-9849-35acde68f3a3" containerName="payload-pre-processing"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897196    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f1c467c6-c1dc-4095-9849-35acde68f3a3" containerName="payload-pre-processing"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897204    5704 cpu_manager.go:414] "containerMap: removing container" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerName="istio-proxy"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897208    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerName="istio-proxy"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897214    5704 cpu_manager.go:414] "containerMap: removing container" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerName="istio-proxy"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897218    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerName="istio-proxy"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897308    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="57bc5590-a521-4cdd-9f55-7667b13bcfba" containerName="payload-pre-processing"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897317    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="bba0064f-500b-4920-88e7-43dc15b56e3e" containerName="main"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897323    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="768c017a-29de-4fdc-8782-6924c6740b21" containerName="maas-api"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897328    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="17b9a105-e07d-4d0f-8c64-4e4d6c7bc69b" containerName="payload-pre-processing"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897334    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="20d434f7-33e5-4136-9114-43d281037737" containerName="payload-processing"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897341    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d7b5569-0635-4061-b3d7-eb9890ad2184" containerName="main"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897348    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="1803662c-638a-4442-b019-1ef97209f2a8" containerName="payload-processing"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897353    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="f1c467c6-c1dc-4095-9849-35acde68f3a3" containerName="payload-pre-processing"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897359    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="902a14d3-7e9d-448c-a951-465010109b2c" containerName="istio-proxy"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897364    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerName="istio-proxy"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897369    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="139cc6a4-643f-41ee-a8c2-eaf2177f4052" containerName="istio-proxy"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897376    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="e84096c6-01ea-407c-8d70-46f7b14bde3f" containerName="main"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897382    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="902a14d3-7e9d-448c-a951-465010109b2c" containerName="istio-proxy"
Oct 30 19:59:32.897408 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.897389    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="0f628790-151a-4760-9dfd-62f0ca9c87e4" containerName="payload-processing"
Oct 30 19:59:32.902251 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.902234    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:32.987337 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.987310    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-f77d05\"/\"agc-cred-930266f4583d\""
Oct 30 19:59:32.997773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.997750    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-f77d05\"/\"payload-processing-external-model-e2e--d1ed065d-dockercfg-5s4x6\""
Oct 30 19:59:32.997773 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.997770    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f77d05\"/\"kube-root-ca.crt\""
Oct 30 19:59:32.998091 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.998077    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f77d05\"/\"payload-processing-external-model-plugins-e2e-extmod-b-f77d05\""
Oct 30 19:59:32.998170 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.998158    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f77d05\"/\"openshift-service-ca.crt\""
Oct 30 19:59:32.998219 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:32.998173    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-b-f77d05\"/\"routing-overlay\""
Oct 30 19:59:33.000207 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.000191    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"]
Oct 30 19:59:33.006926 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.006908    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:33.006999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.006936    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:33.007082 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.007065    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spwln\" (UniqueName: \"kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-kube-api-access-spwln\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:33.007139 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.007125    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:33.108470 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.108445    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spwln\" (UniqueName: \"kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-kube-api-access-spwln\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:33.108633 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.108519    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:33.108633 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.108618    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:33.108726 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.108654    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:33.109341 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.109317    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:33.109477 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.109356    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:33.111378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.111358    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:33.166237 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.166182    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-spwln\" (UniqueName: \"kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-kube-api-access-spwln\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:33.215863 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.215840    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:33.384081 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:59:33.384023    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 19:59:33.615903 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:33.615835    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"]
Oct 30 19:59:33.618491 ip-10-0-139-33 kubenswrapper[5704]: W1030 19:59:33.618462    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6db8a91c_6926_4df9_8c74_453363556742.slice/crio-6836050ee42e4e30d4752f744ae4877111b3095e8814790c138a825f9dbc9be7 WatchSource:0}: Error finding container 6836050ee42e4e30d4752f744ae4877111b3095e8814790c138a825f9dbc9be7: Status 404 returned error can't find the container with id 6836050ee42e4e30d4752f744ae4877111b3095e8814790c138a825f9dbc9be7
Oct 30 19:59:34.528367 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:34.528331    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6" event={"ID":"6db8a91c-6926-4df9-8c74-453363556742","Type":"ContainerStarted","Data":"8c4ce6cec246120021908732acffcfbf28ad26254abc89e0f8b112c21f6d11b7"}
Oct 30 19:59:34.528558 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:34.528392    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6" event={"ID":"6db8a91c-6926-4df9-8c74-453363556742","Type":"ContainerStarted","Data":"6836050ee42e4e30d4752f744ae4877111b3095e8814790c138a825f9dbc9be7"}
Oct 30 19:59:34.528558 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:34.528423    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:34.558537 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:34.558491    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:34.641339 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:34.641287    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6" podStartSLOduration=2.641272354 podStartE2EDuration="2.641272354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:59:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:59:34.639576118 +0000 UTC m=+4347.585744021" watchObservedRunningTime="2026-10-30 19:59:34.641272354 +0000 UTC m=+4347.587440262"
Oct 30 19:59:34.760130 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:34.758410    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:34.887684 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:34.887590    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"]
Oct 30 19:59:34.887877 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:34.887809    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx" podUID="25293f2e-cd91-42b5-b7b6-8d3568f1bb32" containerName="istio-proxy" containerID="cri-o://f60e1c28a175c38ce53bff5b3dff63e61a308162b9420701db457ce90f2ab092" gracePeriod=30
Oct 30 19:59:34.987136 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:34.987106    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:35.285386 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.285355    5704 patch_prober.go:29] interesting pod/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Oct 30 19:59:35.285559 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.285408    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx" podUID="25293f2e-cd91-42b5-b7b6-8d3568f1bb32" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Oct 30 19:59:35.367913 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.367888    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 19:59:35.452413 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.452387    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"]
Oct 30 19:59:35.452570 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.452557    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt" podUID="b15a95b6-341d-407c-b8ce-d88a5cf155de" containerName="maas-api" containerID="cri-o://ebc9dd33cff979d8af14bde7deee9016374d7cb0bb6e05dd1f8b5d2c68cd57a7" gracePeriod=30
Oct 30 19:59:35.596505 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.596460    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:35.733893 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.733868    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:59:35.835291 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.835202    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b15a95b6-341d-407c-b8ce-d88a5cf155de-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b15a95b6-341d-407c-b8ce-d88a5cf155de-openshift-service-ca\") pod \"b15a95b6-341d-407c-b8ce-d88a5cf155de\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") "
Oct 30 19:59:35.835291 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.835281    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-metrics-certs\") pod \"b15a95b6-341d-407c-b8ce-d88a5cf155de\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") "
Oct 30 19:59:35.835533 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.835314    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b15a95b6-341d-407c-b8ce-d88a5cf155de-kube-api-access-k6vz4\" (UniqueName: \"kubernetes.io/projected/b15a95b6-341d-407c-b8ce-d88a5cf155de-kube-api-access-k6vz4\") pod \"b15a95b6-341d-407c-b8ce-d88a5cf155de\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") "
Oct 30 19:59:35.835533 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.835480    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-maas-api-tls\") pod \"b15a95b6-341d-407c-b8ce-d88a5cf155de\" (UID: \"b15a95b6-341d-407c-b8ce-d88a5cf155de\") "
Oct 30 19:59:35.835645 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.835605    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b15a95b6-341d-407c-b8ce-d88a5cf155de-openshift-service-ca" pod "b15a95b6-341d-407c-b8ce-d88a5cf155de" (UID: "b15a95b6-341d-407c-b8ce-d88a5cf155de"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:35.835862 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.835845    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b15a95b6-341d-407c-b8ce-d88a5cf155de-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:35.837640 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.837611    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-maas-api-tls" pod "b15a95b6-341d-407c-b8ce-d88a5cf155de" (UID: "b15a95b6-341d-407c-b8ce-d88a5cf155de"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:35.837755 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.837638    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-metrics-certs" pod "b15a95b6-341d-407c-b8ce-d88a5cf155de" (UID: "b15a95b6-341d-407c-b8ce-d88a5cf155de"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:35.837755 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.837653    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b15a95b6-341d-407c-b8ce-d88a5cf155de-kube-api-access-k6vz4" pod "b15a95b6-341d-407c-b8ce-d88a5cf155de" (UID: "b15a95b6-341d-407c-b8ce-d88a5cf155de"). InnerVolumeSpecName "kube-api-access-k6vz4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:35.936787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.936755    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:35.936787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.936782    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6vz4\" (UniqueName: \"kubernetes.io/projected/b15a95b6-341d-407c-b8ce-d88a5cf155de-kube-api-access-k6vz4\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:35.936787 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:35.936792    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b15a95b6-341d-407c-b8ce-d88a5cf155de-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:36.540819 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:36.540782    5704 generic.go:356] "Generic (PLEG): container finished" podID="b15a95b6-341d-407c-b8ce-d88a5cf155de" containerID="ebc9dd33cff979d8af14bde7deee9016374d7cb0bb6e05dd1f8b5d2c68cd57a7" exitCode=0
Oct 30 19:59:36.540999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:36.540828    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"
Oct 30 19:59:36.540999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:36.540869    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt" event={"ID":"b15a95b6-341d-407c-b8ce-d88a5cf155de","Type":"ContainerDied","Data":"ebc9dd33cff979d8af14bde7deee9016374d7cb0bb6e05dd1f8b5d2c68cd57a7"}
Oct 30 19:59:36.540999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:36.540924    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt" event={"ID":"b15a95b6-341d-407c-b8ce-d88a5cf155de","Type":"ContainerDied","Data":"90f7634a281f80a8a6331c6d3a1a3c8ea6b733cab447311ef86d6217ceccde9c"}
Oct 30 19:59:36.540999 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:36.540946    5704 scope.go:122] "RemoveContainer" containerID="ebc9dd33cff979d8af14bde7deee9016374d7cb0bb6e05dd1f8b5d2c68cd57a7"
Oct 30 19:59:36.550881 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:36.550865    5704 scope.go:122] "RemoveContainer" containerID="ebc9dd33cff979d8af14bde7deee9016374d7cb0bb6e05dd1f8b5d2c68cd57a7"
Oct 30 19:59:36.551144 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:59:36.551127    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ebc9dd33cff979d8af14bde7deee9016374d7cb0bb6e05dd1f8b5d2c68cd57a7\": container with ID starting with ebc9dd33cff979d8af14bde7deee9016374d7cb0bb6e05dd1f8b5d2c68cd57a7 not found: ID does not exist" containerID="ebc9dd33cff979d8af14bde7deee9016374d7cb0bb6e05dd1f8b5d2c68cd57a7"
Oct 30 19:59:36.551201 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:36.551151    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ebc9dd33cff979d8af14bde7deee9016374d7cb0bb6e05dd1f8b5d2c68cd57a7"} err="failed to get container status \"ebc9dd33cff979d8af14bde7deee9016374d7cb0bb6e05dd1f8b5d2c68cd57a7\": rpc error: code = NotFound desc = could not find container \"ebc9dd33cff979d8af14bde7deee9016374d7cb0bb6e05dd1f8b5d2c68cd57a7\": container with ID starting with ebc9dd33cff979d8af14bde7deee9016374d7cb0bb6e05dd1f8b5d2c68cd57a7 not found: ID does not exist"
Oct 30 19:59:36.705788 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:36.705755    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"]
Oct 30 19:59:36.710427 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:36.710397    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:36.710730 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:36.710711    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-2301cb-7767cbfdcf-zb9vt"]
Oct 30 19:59:37.071980 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:37.071940    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:37.178843 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:37.178804    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b15a95b6-341d-407c-b8ce-d88a5cf155de" path="/var/lib/kubelet/pods/b15a95b6-341d-407c-b8ce-d88a5cf155de/volumes"
Oct 30 19:59:37.359378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:37.359299    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:38.009094 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:38.009059    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:38.190217 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:38.190172    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:38.391023 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:38.390924    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:38.577757 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:38.577725    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:39.159002 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:39.158971    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:40.143610 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.143581    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:59:40.280828 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.280802    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-credential-socket\") pod \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") "
Oct 30 19:59:40.280996 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.280872    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-data\") pod \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") "
Oct 30 19:59:40.280996 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.280902    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-socket\") pod \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") "
Oct 30 19:59:40.280996 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.280965    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istiod-ca-cert\") pod \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") "
Oct 30 19:59:40.281191 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281014    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-envoy\") pod \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") "
Oct 30 19:59:40.281191 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281107    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-kube-api-access-ww6kw\" (UniqueName: \"kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-kube-api-access-ww6kw\") pod \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") "
Oct 30 19:59:40.281191 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281152    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-credential-socket" pod "25293f2e-cd91-42b5-b7b6-8d3568f1bb32" (UID: "25293f2e-cd91-42b5-b7b6-8d3568f1bb32"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:40.281191 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281157    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-certs\") pod \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") "
Oct 30 19:59:40.281400 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281190    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-data" pod "25293f2e-cd91-42b5-b7b6-8d3568f1bb32" (UID: "25293f2e-cd91-42b5-b7b6-8d3568f1bb32"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:40.281400 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281206    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-token\" (UniqueName: \"kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-token\") pod \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") "
Oct 30 19:59:40.281518 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281395    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-podinfo\") pod \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\" (UID: \"25293f2e-cd91-42b5-b7b6-8d3568f1bb32\") "
Oct 30 19:59:40.281518 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281404    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-socket" pod "25293f2e-cd91-42b5-b7b6-8d3568f1bb32" (UID: "25293f2e-cd91-42b5-b7b6-8d3568f1bb32"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:40.281708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281525    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-certs" pod "25293f2e-cd91-42b5-b7b6-8d3568f1bb32" (UID: "25293f2e-cd91-42b5-b7b6-8d3568f1bb32"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:40.281708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281541    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istiod-ca-cert" pod "25293f2e-cd91-42b5-b7b6-8d3568f1bb32" (UID: "25293f2e-cd91-42b5-b7b6-8d3568f1bb32"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:40.281957 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281863    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:40.281957 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281890    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:40.281957 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281906    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:40.281957 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281919    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:40.281957 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.281933    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:40.283388 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.283367    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-envoy" pod "25293f2e-cd91-42b5-b7b6-8d3568f1bb32" (UID: "25293f2e-cd91-42b5-b7b6-8d3568f1bb32"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:40.283463 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.283449    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-kube-api-access-ww6kw" pod "25293f2e-cd91-42b5-b7b6-8d3568f1bb32" (UID: "25293f2e-cd91-42b5-b7b6-8d3568f1bb32"). InnerVolumeSpecName "kube-api-access-ww6kw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:40.283708 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.283685    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-podinfo" pod "25293f2e-cd91-42b5-b7b6-8d3568f1bb32" (UID: "25293f2e-cd91-42b5-b7b6-8d3568f1bb32"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:59:40.283796 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.283688    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-token" pod "25293f2e-cd91-42b5-b7b6-8d3568f1bb32" (UID: "25293f2e-cd91-42b5-b7b6-8d3568f1bb32"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:40.382794 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.382753    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:40.382794 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.382780    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ww6kw\" (UniqueName: \"kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-kube-api-access-ww6kw\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:40.382794 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.382790    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:40.382794 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.382799    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25293f2e-cd91-42b5-b7b6-8d3568f1bb32-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:40.563667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.563566    5704 generic.go:356] "Generic (PLEG): container finished" podID="25293f2e-cd91-42b5-b7b6-8d3568f1bb32" containerID="f60e1c28a175c38ce53bff5b3dff63e61a308162b9420701db457ce90f2ab092" exitCode=0
Oct 30 19:59:40.563667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.563606    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx" event={"ID":"25293f2e-cd91-42b5-b7b6-8d3568f1bb32","Type":"ContainerDied","Data":"f60e1c28a175c38ce53bff5b3dff63e61a308162b9420701db457ce90f2ab092"}
Oct 30 19:59:40.563667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.563633    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"
Oct 30 19:59:40.563667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.563652    5704 scope.go:122] "RemoveContainer" containerID="f60e1c28a175c38ce53bff5b3dff63e61a308162b9420701db457ce90f2ab092"
Oct 30 19:59:40.563997 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.563643    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx" event={"ID":"25293f2e-cd91-42b5-b7b6-8d3568f1bb32","Type":"ContainerDied","Data":"6b1cb552a6c4bc3f7a1941ae3b973c131103dc99f3325f12d1bd6500e99a2d92"}
Oct 30 19:59:40.573327 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.573309    5704 scope.go:122] "RemoveContainer" containerID="f60e1c28a175c38ce53bff5b3dff63e61a308162b9420701db457ce90f2ab092"
Oct 30 19:59:40.573577 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:59:40.573558    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f60e1c28a175c38ce53bff5b3dff63e61a308162b9420701db457ce90f2ab092\": container with ID starting with f60e1c28a175c38ce53bff5b3dff63e61a308162b9420701db457ce90f2ab092 not found: ID does not exist" containerID="f60e1c28a175c38ce53bff5b3dff63e61a308162b9420701db457ce90f2ab092"
Oct 30 19:59:40.573627 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.573587    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f60e1c28a175c38ce53bff5b3dff63e61a308162b9420701db457ce90f2ab092"} err="failed to get container status \"f60e1c28a175c38ce53bff5b3dff63e61a308162b9420701db457ce90f2ab092\": rpc error: code = NotFound desc = could not find container \"f60e1c28a175c38ce53bff5b3dff63e61a308162b9420701db457ce90f2ab092\": container with ID starting with f60e1c28a175c38ce53bff5b3dff63e61a308162b9420701db457ce90f2ab092 not found: ID does not exist"
Oct 30 19:59:40.725157 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.725127    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"]
Oct 30 19:59:40.730321 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:40.730302    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-22fb18-openshift-default-d9c87fb8f-vw2wx"]
Oct 30 19:59:41.178923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:41.178890    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="25293f2e-cd91-42b5-b7b6-8d3568f1bb32" path="/var/lib/kubelet/pods/25293f2e-cd91-42b5-b7b6-8d3568f1bb32/volumes"
Oct 30 19:59:42.279539 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:42.279508    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:42.513703 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:42.513668    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:43.434419 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:43.434381    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:45.537284 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:45.537250    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 19:59:46.035923 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:46.035889    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:46.224641 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:46.224604    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:46.437767 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:46.437683    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:46.625528 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:46.625492    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:46.819688 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:46.819652    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:47.034379 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.034342    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:47.310527 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.310497    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"]
Oct 30 19:59:47.310711 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.310692    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j" podUID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerName="istio-proxy" containerID="cri-o://728aea0ad48d59e38832e4e768f7080a673a7b8ccba0e1ced675f2e948d98fd3" gracePeriod=30
Oct 30 19:59:47.391770 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.391721    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:47.450241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.450202    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"]
Oct 30 19:59:47.450438 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.450401    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl" podUID="c9af97f5-65b8-415a-847c-ee8fb9e67da5" containerName="maas-api" containerID="cri-o://36ca8f7d28461dbed5bc293fe8a50a0600feb2818c990e46ed7069245ff7cc8e" gracePeriod=30
Oct 30 19:59:47.590244 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.590207    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:47.605333 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.605303    5704 generic.go:356] "Generic (PLEG): container finished" podID="c9af97f5-65b8-415a-847c-ee8fb9e67da5" containerID="36ca8f7d28461dbed5bc293fe8a50a0600feb2818c990e46ed7069245ff7cc8e" exitCode=0
Oct 30 19:59:47.605469 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.605371    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl" event={"ID":"c9af97f5-65b8-415a-847c-ee8fb9e67da5","Type":"ContainerDied","Data":"36ca8f7d28461dbed5bc293fe8a50a0600feb2818c990e46ed7069245ff7cc8e"}
Oct 30 19:59:47.710008 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.709984    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:59:47.753341 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.753314    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c9af97f5-65b8-415a-847c-ee8fb9e67da5-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9af97f5-65b8-415a-847c-ee8fb9e67da5-openshift-service-ca\") pod \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") "
Oct 30 19:59:47.753461 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.753365    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-maas-api-tls\") pod \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") "
Oct 30 19:59:47.753461 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.753452    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c9af97f5-65b8-415a-847c-ee8fb9e67da5-kube-api-access-wpg5g\" (UniqueName: \"kubernetes.io/projected/c9af97f5-65b8-415a-847c-ee8fb9e67da5-kube-api-access-wpg5g\") pod \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") "
Oct 30 19:59:47.753582 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.753480    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-metrics-certs\") pod \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\" (UID: \"c9af97f5-65b8-415a-847c-ee8fb9e67da5\") "
Oct 30 19:59:47.753719 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.753692    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9af97f5-65b8-415a-847c-ee8fb9e67da5-openshift-service-ca" pod "c9af97f5-65b8-415a-847c-ee8fb9e67da5" (UID: "c9af97f5-65b8-415a-847c-ee8fb9e67da5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:47.753822 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.753800    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c9af97f5-65b8-415a-847c-ee8fb9e67da5-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:47.755633 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.755605    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-maas-api-tls" pod "c9af97f5-65b8-415a-847c-ee8fb9e67da5" (UID: "c9af97f5-65b8-415a-847c-ee8fb9e67da5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:47.755633 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.755623    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9af97f5-65b8-415a-847c-ee8fb9e67da5-kube-api-access-wpg5g" pod "c9af97f5-65b8-415a-847c-ee8fb9e67da5" (UID: "c9af97f5-65b8-415a-847c-ee8fb9e67da5"). InnerVolumeSpecName "kube-api-access-wpg5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:47.755784 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.755706    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-metrics-certs" pod "c9af97f5-65b8-415a-847c-ee8fb9e67da5" (UID: "c9af97f5-65b8-415a-847c-ee8fb9e67da5"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 19:59:47.854591 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.854514    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:47.854591 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.854543    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpg5g\" (UniqueName: \"kubernetes.io/projected/c9af97f5-65b8-415a-847c-ee8fb9e67da5-kube-api-access-wpg5g\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:47.854591 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:47.854552    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c9af97f5-65b8-415a-847c-ee8fb9e67da5-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:48.613193 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:48.613162    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"
Oct 30 19:59:48.613193 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:48.613185    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl" event={"ID":"c9af97f5-65b8-415a-847c-ee8fb9e67da5","Type":"ContainerDied","Data":"1dccc39a46597208e94fcd38d001df42bafdd31a7601a1bca75ce1a292cdf732"}
Oct 30 19:59:48.613420 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:48.613245    5704 scope.go:122] "RemoveContainer" containerID="36ca8f7d28461dbed5bc293fe8a50a0600feb2818c990e46ed7069245ff7cc8e"
Oct 30 19:59:48.766514 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:48.766483    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"]
Oct 30 19:59:48.812174 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:48.812149    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-c01685-587f68f9f5-wr8bl"]
Oct 30 19:59:49.082959 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:49.082923    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:49.179287 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:49.179258    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c9af97f5-65b8-415a-847c-ee8fb9e67da5" path="/var/lib/kubelet/pods/c9af97f5-65b8-415a-847c-ee8fb9e67da5/volumes"
Oct 30 19:59:49.275516 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:49.275489    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:49.456435 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:49.456352    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:49.958667 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:49.958632    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:52.568574 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.568550    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:59:52.602241 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.602175    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-podinfo\") pod \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") "
Oct 30 19:59:52.602378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.602240    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-data\") pod \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") "
Oct 30 19:59:52.602378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.602291    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-token\" (UniqueName: \"kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-token\") pod \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") "
Oct 30 19:59:52.602378 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.602339    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-kube-api-access-w9vgk\" (UniqueName: \"kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-kube-api-access-w9vgk\") pod \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") "
Oct 30 19:59:52.602551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.602416    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-certs\") pod \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") "
Oct 30 19:59:52.602551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.602449    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-credential-socket\") pod \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") "
Oct 30 19:59:52.602551 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.602528    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-envoy\") pod \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") "
Oct 30 19:59:52.602717 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.602582    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-socket\") pod \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") "
Oct 30 19:59:52.602717 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.602628    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istiod-ca-cert\") pod \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\" (UID: \"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270\") "
Oct 30 19:59:52.602717 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.602665    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-certs" pod "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" (UID: "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:52.602865 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.602752    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-credential-socket" pod "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" (UID: "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:52.602865 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.602818    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-data" pod "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" (UID: "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:52.603106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.603081    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.603106 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.603106    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.603352 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.603120    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.603352 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.603220    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-socket" pod "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" (UID: "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:52.603352 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.603285    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istiod-ca-cert" pod "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" (UID: "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 19:59:52.604494 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.604467    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-kube-api-access-w9vgk" pod "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" (UID: "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270"). InnerVolumeSpecName "kube-api-access-w9vgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:52.604494 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.604485    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-podinfo" pod "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" (UID: "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 19:59:52.604677 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.604542    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-token" pod "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" (UID: "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 19:59:52.604752 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.604737    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-envoy" pod "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" (UID: "eb4aa4c2-580b-4c9a-a6bf-5e4de536f270"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 19:59:52.636221 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.636195    5704 generic.go:356] "Generic (PLEG): container finished" podID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerID="728aea0ad48d59e38832e4e768f7080a673a7b8ccba0e1ced675f2e948d98fd3" exitCode=0
Oct 30 19:59:52.636312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.636246    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"
Oct 30 19:59:52.636312 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.636280    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j" event={"ID":"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270","Type":"ContainerDied","Data":"728aea0ad48d59e38832e4e768f7080a673a7b8ccba0e1ced675f2e948d98fd3"}
Oct 30 19:59:52.636392 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.636333    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j" event={"ID":"eb4aa4c2-580b-4c9a-a6bf-5e4de536f270","Type":"ContainerDied","Data":"c3cb4e47a9779f1a10f28bfce02e050bcf8f9f1b4871ff028cccc4d26d7ed433"}
Oct 30 19:59:52.636392 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.636357    5704 scope.go:122] "RemoveContainer" containerID="728aea0ad48d59e38832e4e768f7080a673a7b8ccba0e1ced675f2e948d98fd3"
Oct 30 19:59:52.648058 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.646444    5704 scope.go:122] "RemoveContainer" containerID="ca6aa4e2f18fd15e4e12d15b68ae016b72b9919ffaa7d9580feda18e9491392b"
Oct 30 19:59:52.662088 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.662069    5704 scope.go:122] "RemoveContainer" containerID="728aea0ad48d59e38832e4e768f7080a673a7b8ccba0e1ced675f2e948d98fd3"
Oct 30 19:59:52.662363 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:59:52.662343    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"728aea0ad48d59e38832e4e768f7080a673a7b8ccba0e1ced675f2e948d98fd3\": container with ID starting with 728aea0ad48d59e38832e4e768f7080a673a7b8ccba0e1ced675f2e948d98fd3 not found: ID does not exist" containerID="728aea0ad48d59e38832e4e768f7080a673a7b8ccba0e1ced675f2e948d98fd3"
Oct 30 19:59:52.662463 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.662372    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"728aea0ad48d59e38832e4e768f7080a673a7b8ccba0e1ced675f2e948d98fd3"} err="failed to get container status \"728aea0ad48d59e38832e4e768f7080a673a7b8ccba0e1ced675f2e948d98fd3\": rpc error: code = NotFound desc = could not find container \"728aea0ad48d59e38832e4e768f7080a673a7b8ccba0e1ced675f2e948d98fd3\": container with ID starting with 728aea0ad48d59e38832e4e768f7080a673a7b8ccba0e1ced675f2e948d98fd3 not found: ID does not exist"
Oct 30 19:59:52.662463 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.662391    5704 scope.go:122] "RemoveContainer" containerID="ca6aa4e2f18fd15e4e12d15b68ae016b72b9919ffaa7d9580feda18e9491392b"
Oct 30 19:59:52.662629 ip-10-0-139-33 kubenswrapper[5704]: E1030 19:59:52.662604    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca6aa4e2f18fd15e4e12d15b68ae016b72b9919ffaa7d9580feda18e9491392b\": container with ID starting with ca6aa4e2f18fd15e4e12d15b68ae016b72b9919ffaa7d9580feda18e9491392b not found: ID does not exist" containerID="ca6aa4e2f18fd15e4e12d15b68ae016b72b9919ffaa7d9580feda18e9491392b"
Oct 30 19:59:52.662690 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.662639    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca6aa4e2f18fd15e4e12d15b68ae016b72b9919ffaa7d9580feda18e9491392b"} err="failed to get container status \"ca6aa4e2f18fd15e4e12d15b68ae016b72b9919ffaa7d9580feda18e9491392b\": rpc error: code = NotFound desc = could not find container \"ca6aa4e2f18fd15e4e12d15b68ae016b72b9919ffaa7d9580feda18e9491392b\": container with ID starting with ca6aa4e2f18fd15e4e12d15b68ae016b72b9919ffaa7d9580feda18e9491392b not found: ID does not exist"
Oct 30 19:59:52.703601 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.703567    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.703601 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.703598    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.703742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.703613    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.703742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.703625    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.703742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.703637    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9vgk\" (UniqueName: \"kubernetes.io/projected/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-kube-api-access-w9vgk\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.703742 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.703651    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 19:59:52.833838 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.833804    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"]
Oct 30 19:59:52.838082 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:52.838055    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-9275c3-openshift-default-56ffdfd665-z4j9j"]
Oct 30 19:59:53.179173 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:53.179140    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" path="/var/lib/kubelet/pods/eb4aa4c2-580b-4c9a-a6bf-5e4de536f270/volumes"
Oct 30 19:59:54.222617 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:54.222533    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:56.854985 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:56.854958    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:57.054430 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:57.054395    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:57.297679 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:57.297630    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:57.726613 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:57.726531    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 19:59:59.068439 ip-10-0-139-33 kubenswrapper[5704]: I1030 19:59:59.068407    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:00.184836 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.184808    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"]
Oct 30 20:00:00.185362 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185289    5704 cpu_manager.go:414] "containerMap: removing container" podUID="902a14d3-7e9d-448c-a951-465010109b2c" containerName="istio-proxy"
Oct 30 20:00:00.185362 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185302    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="902a14d3-7e9d-448c-a951-465010109b2c" containerName="istio-proxy"
Oct 30 20:00:00.185362 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185314    5704 cpu_manager.go:414] "containerMap: removing container" podUID="25293f2e-cd91-42b5-b7b6-8d3568f1bb32" containerName="istio-proxy"
Oct 30 20:00:00.185362 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185319    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="25293f2e-cd91-42b5-b7b6-8d3568f1bb32" containerName="istio-proxy"
Oct 30 20:00:00.185362 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185339    5704 cpu_manager.go:414] "containerMap: removing container" podUID="b15a95b6-341d-407c-b8ce-d88a5cf155de" containerName="maas-api"
Oct 30 20:00:00.185362 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185344    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b15a95b6-341d-407c-b8ce-d88a5cf155de" containerName="maas-api"
Oct 30 20:00:00.185362 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185353    5704 cpu_manager.go:414] "containerMap: removing container" podUID="c9af97f5-65b8-415a-847c-ee8fb9e67da5" containerName="maas-api"
Oct 30 20:00:00.185362 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185358    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c9af97f5-65b8-415a-847c-ee8fb9e67da5" containerName="maas-api"
Oct 30 20:00:00.185628 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185374    5704 cpu_manager.go:414] "containerMap: removing container" podUID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerName="istio-proxy"
Oct 30 20:00:00.185628 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185379    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerName="istio-proxy"
Oct 30 20:00:00.185628 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185437    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerName="istio-proxy"
Oct 30 20:00:00.185628 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185447    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerName="istio-proxy"
Oct 30 20:00:00.185628 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185455    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="25293f2e-cd91-42b5-b7b6-8d3568f1bb32" containerName="istio-proxy"
Oct 30 20:00:00.185628 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185463    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="b15a95b6-341d-407c-b8ce-d88a5cf155de" containerName="maas-api"
Oct 30 20:00:00.185628 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.185468    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="c9af97f5-65b8-415a-847c-ee8fb9e67da5" containerName="maas-api"
Oct 30 20:00:00.190098 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.190068    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"
Oct 30 20:00:00.238761 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.238738    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-b8jtx\""
Oct 30 20:00:00.278473 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.278439    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4015dc19-7986-49c4-a744-2c619f68d81f-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889840-4lcbv\" (UID: \"4015dc19-7986-49c4-a744-2c619f68d81f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"
Oct 30 20:00:00.278764 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.278736    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7qsg\" (UniqueName: \"kubernetes.io/projected/4015dc19-7986-49c4-a744-2c619f68d81f-kube-api-access-t7qsg\") pod \"maas-api-key-cleanup-29889840-4lcbv\" (UID: \"4015dc19-7986-49c4-a744-2c619f68d81f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"
Oct 30 20:00:00.286810 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.286787    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"]
Oct 30 20:00:00.379846 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.379815    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4015dc19-7986-49c4-a744-2c619f68d81f-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889840-4lcbv\" (UID: \"4015dc19-7986-49c4-a744-2c619f68d81f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"
Oct 30 20:00:00.380009 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.379863    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7qsg\" (UniqueName: \"kubernetes.io/projected/4015dc19-7986-49c4-a744-2c619f68d81f-kube-api-access-t7qsg\") pod \"maas-api-key-cleanup-29889840-4lcbv\" (UID: \"4015dc19-7986-49c4-a744-2c619f68d81f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"
Oct 30 20:00:00.380483 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.380464    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4015dc19-7986-49c4-a744-2c619f68d81f-maas-api-service-ca\") pod \"maas-api-key-cleanup-29889840-4lcbv\" (UID: \"4015dc19-7986-49c4-a744-2c619f68d81f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"
Oct 30 20:00:00.437846 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.437771    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7qsg\" (UniqueName: \"kubernetes.io/projected/4015dc19-7986-49c4-a744-2c619f68d81f-kube-api-access-t7qsg\") pod \"maas-api-key-cleanup-29889840-4lcbv\" (UID: \"4015dc19-7986-49c4-a744-2c619f68d81f\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"
Oct 30 20:00:00.502451 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.502414    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"
Oct 30 20:00:00.874515 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:00.874491    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"]
Oct 30 20:00:00.877348 ip-10-0-139-33 kubenswrapper[5704]: W1030 20:00:00.877307    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4015dc19_7986_49c4_a744_2c619f68d81f.slice/crio-ea9fd13cec7a4cdfc30ce4ac7c1a8fe680327526c722d7e6ddfa652caf647916 WatchSource:0}: Error finding container ea9fd13cec7a4cdfc30ce4ac7c1a8fe680327526c722d7e6ddfa652caf647916: Status 404 returned error can't find the container with id ea9fd13cec7a4cdfc30ce4ac7c1a8fe680327526c722d7e6ddfa652caf647916
Oct 30 20:00:01.683440 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:01.683407    5704 generic.go:356] "Generic (PLEG): container finished" podID="4015dc19-7986-49c4-a744-2c619f68d81f" containerID="178ab5f5b692ced92b78830ef40f8d8e2dbc629f90973ee9d5f98d0b0011928f" exitCode=60
Oct 30 20:00:01.683877 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:01.683466    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv" event={"ID":"4015dc19-7986-49c4-a744-2c619f68d81f","Type":"ContainerDied","Data":"178ab5f5b692ced92b78830ef40f8d8e2dbc629f90973ee9d5f98d0b0011928f"}
Oct 30 20:00:01.683877 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:01.683512    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv" event={"ID":"4015dc19-7986-49c4-a744-2c619f68d81f","Type":"ContainerStarted","Data":"ea9fd13cec7a4cdfc30ce4ac7c1a8fe680327526c722d7e6ddfa652caf647916"}
Oct 30 20:00:01.683877 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:01.683751    5704 scope.go:122] "RemoveContainer" containerID="178ab5f5b692ced92b78830ef40f8d8e2dbc629f90973ee9d5f98d0b0011928f"
Oct 30 20:00:02.691332 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:02.691262    5704 generic.go:356] "Generic (PLEG): container finished" podID="4015dc19-7986-49c4-a744-2c619f68d81f" containerID="a5c51a133a62a7fadeac543e11f3c8e2c173d9ad36e8ae9e1c36c357692724d0" exitCode=60
Oct 30 20:00:02.691332 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:02.691327    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv" event={"ID":"4015dc19-7986-49c4-a744-2c619f68d81f","Type":"ContainerDied","Data":"a5c51a133a62a7fadeac543e11f3c8e2c173d9ad36e8ae9e1c36c357692724d0"}
Oct 30 20:00:02.691893 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:02.691387    5704 scope.go:122] "RemoveContainer" containerID="178ab5f5b692ced92b78830ef40f8d8e2dbc629f90973ee9d5f98d0b0011928f"
Oct 30 20:00:02.691893 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:02.691720    5704 scope.go:122] "RemoveContainer" containerID="a5c51a133a62a7fadeac543e11f3c8e2c173d9ad36e8ae9e1c36c357692724d0"
Oct 30 20:00:02.692055 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:02.692010    5704 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889840-4lcbv_odh-ai-gateway-infra(4015dc19-7986-49c4-a744-2c619f68d81f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv" podUID="4015dc19-7986-49c4-a744-2c619f68d81f"
Oct 30 20:00:02.830263 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:02.830172    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:03.383533 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:03.383496    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 20:00:03.698399 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:03.698320    5704 scope.go:122] "RemoveContainer" containerID="a5c51a133a62a7fadeac543e11f3c8e2c173d9ad36e8ae9e1c36c357692724d0"
Oct 30 20:00:03.698823 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:03.698641    5704 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-29889840-4lcbv_odh-ai-gateway-infra(4015dc19-7986-49c4-a744-2c619f68d81f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv" podUID="4015dc19-7986-49c4-a744-2c619f68d81f"
Oct 30 20:00:04.082913 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:04.082871    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:07.235361 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:07.235326    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 20:00:08.879327 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:08.879229    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:09.747091 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:09.747026    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"]
Oct 30 20:00:10.038903 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:10.038815    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:10.272205 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:10.272166    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:13.583830 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:13.583792    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:14.748349 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:14.748308    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79" podUID="b8e3bcdb-a865-428a-8771-6ed558e34369" containerName="payload-pre-processing" containerID="cri-o://4b902dee181dd11b7ed56adb77e8b0956169eace85c2a69cc298bace43f5756f" gracePeriod=25
Oct 30 20:00:16.172386 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:16.172357    5704 scope.go:122] "RemoveContainer" containerID="a5c51a133a62a7fadeac543e11f3c8e2c173d9ad36e8ae9e1c36c357692724d0"
Oct 30 20:00:16.765896 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:16.765860    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv" event={"ID":"4015dc19-7986-49c4-a744-2c619f68d81f","Type":"ContainerStarted","Data":"39057cb6eb0779005f3af4524dcf36d02123b5354c5b14a859465f59f48d7aba"}
Oct 30 20:00:16.887315 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:16.887262    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv" podStartSLOduration=16.887248903 podStartE2EDuration="16.887248903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:16.886242467 +0000 UTC m=+4389.832410370" watchObservedRunningTime="2026-10-30 20:00:16.887248903 +0000 UTC m=+4389.833416828"
Oct 30 20:00:17.774873 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:17.774839    5704 generic.go:356] "Generic (PLEG): container finished" podID="4015dc19-7986-49c4-a744-2c619f68d81f" containerID="39057cb6eb0779005f3af4524dcf36d02123b5354c5b14a859465f59f48d7aba" exitCode=60
Oct 30 20:00:17.775301 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:17.774912    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv" event={"ID":"4015dc19-7986-49c4-a744-2c619f68d81f","Type":"ContainerDied","Data":"39057cb6eb0779005f3af4524dcf36d02123b5354c5b14a859465f59f48d7aba"}
Oct 30 20:00:17.775301 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:17.774973    5704 scope.go:122] "RemoveContainer" containerID="a5c51a133a62a7fadeac543e11f3c8e2c173d9ad36e8ae9e1c36c357692724d0"
Oct 30 20:00:17.775426 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:17.775318    5704 scope.go:122] "RemoveContainer" containerID="39057cb6eb0779005f3af4524dcf36d02123b5354c5b14a859465f59f48d7aba"
Oct 30 20:00:17.775614 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:17.775594    5704 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-29889840-4lcbv_odh-ai-gateway-infra(4015dc19-7986-49c4-a744-2c619f68d81f)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv" podUID="4015dc19-7986-49c4-a744-2c619f68d81f"
Oct 30 20:00:17.909686 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:17.909658    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"]
Oct 30 20:00:18.522411 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:18.522376    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:18.913929 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:18.913899    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"
Oct 30 20:00:18.956535 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:18.956500    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:19.061343 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:19.061235    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4015dc19-7986-49c4-a744-2c619f68d81f-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4015dc19-7986-49c4-a744-2c619f68d81f-maas-api-service-ca\") pod \"4015dc19-7986-49c4-a744-2c619f68d81f\" (UID: \"4015dc19-7986-49c4-a744-2c619f68d81f\") "
Oct 30 20:00:19.061481 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:19.061405    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4015dc19-7986-49c4-a744-2c619f68d81f-kube-api-access-t7qsg\" (UniqueName: \"kubernetes.io/projected/4015dc19-7986-49c4-a744-2c619f68d81f-kube-api-access-t7qsg\") pod \"4015dc19-7986-49c4-a744-2c619f68d81f\" (UID: \"4015dc19-7986-49c4-a744-2c619f68d81f\") "
Oct 30 20:00:19.061575 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:19.061554    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4015dc19-7986-49c4-a744-2c619f68d81f-maas-api-service-ca" pod "4015dc19-7986-49c4-a744-2c619f68d81f" (UID: "4015dc19-7986-49c4-a744-2c619f68d81f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:19.061809 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:19.061790    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4015dc19-7986-49c4-a744-2c619f68d81f-maas-api-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:19.063538 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:19.063512    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4015dc19-7986-49c4-a744-2c619f68d81f-kube-api-access-t7qsg" pod "4015dc19-7986-49c4-a744-2c619f68d81f" (UID: "4015dc19-7986-49c4-a744-2c619f68d81f"). InnerVolumeSpecName "kube-api-access-t7qsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:19.162969 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:19.162942    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7qsg\" (UniqueName: \"kubernetes.io/projected/4015dc19-7986-49c4-a744-2c619f68d81f-kube-api-access-t7qsg\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:19.787966 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:19.787931    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv" event={"ID":"4015dc19-7986-49c4-a744-2c619f68d81f","Type":"ContainerDied","Data":"ea9fd13cec7a4cdfc30ce4ac7c1a8fe680327526c722d7e6ddfa652caf647916"}
Oct 30 20:00:19.787966 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:19.787964    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"
Oct 30 20:00:19.788224 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:19.787995    5704 scope.go:122] "RemoveContainer" containerID="39057cb6eb0779005f3af4524dcf36d02123b5354c5b14a859465f59f48d7aba"
Oct 30 20:00:19.946309 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:19.946282    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"]
Oct 30 20:00:19.992264 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:19.992237    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29889840-4lcbv"]
Oct 30 20:00:21.177991 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:21.177962    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4015dc19-7986-49c4-a744-2c619f68d81f" path="/var/lib/kubelet/pods/4015dc19-7986-49c4-a744-2c619f68d81f/volumes"
Oct 30 20:00:23.115689 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:23.115656    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79" podUID="b8e3bcdb-a865-428a-8771-6ed558e34369" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:00:23.271589 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:23.271560    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:27.993571 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:27.993535    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:32.586494 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:32.583618    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:33.115836 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:33.115799    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79" podUID="b8e3bcdb-a865-428a-8771-6ed558e34369" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:00:33.382405 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:33.382327    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 20:00:34.502126 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:34.502095    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 20:00:34.866943 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:34.866918    5704 generic.go:356] "Generic (PLEG): container finished" podID="b8e3bcdb-a865-428a-8771-6ed558e34369" containerID="4b902dee181dd11b7ed56adb77e8b0956169eace85c2a69cc298bace43f5756f" exitCode=0
Oct 30 20:00:34.867014 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:34.866958    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79" event={"ID":"b8e3bcdb-a865-428a-8771-6ed558e34369","Type":"ContainerDied","Data":"4b902dee181dd11b7ed56adb77e8b0956169eace85c2a69cc298bace43f5756f"}
Oct 30 20:00:35.011341 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:35.011284    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"
Oct 30 20:00:35.114084 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:35.114022    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b8e3bcdb-a865-428a-8771-6ed558e34369-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e3bcdb-a865-428a-8771-6ed558e34369-plugins-config-volume\") pod \"b8e3bcdb-a865-428a-8771-6ed558e34369\" (UID: \"b8e3bcdb-a865-428a-8771-6ed558e34369\") "
Oct 30 20:00:35.114281 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:35.114118    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b8e3bcdb-a865-428a-8771-6ed558e34369-kube-api-access-7z59v\" (UniqueName: \"kubernetes.io/projected/b8e3bcdb-a865-428a-8771-6ed558e34369-kube-api-access-7z59v\") pod \"b8e3bcdb-a865-428a-8771-6ed558e34369\" (UID: \"b8e3bcdb-a865-428a-8771-6ed558e34369\") "
Oct 30 20:00:35.114427 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:35.114401    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8e3bcdb-a865-428a-8771-6ed558e34369-plugins-config-volume" pod "b8e3bcdb-a865-428a-8771-6ed558e34369" (UID: "b8e3bcdb-a865-428a-8771-6ed558e34369"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:35.116170 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:35.116144    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8e3bcdb-a865-428a-8771-6ed558e34369-kube-api-access-7z59v" pod "b8e3bcdb-a865-428a-8771-6ed558e34369" (UID: "b8e3bcdb-a865-428a-8771-6ed558e34369"). InnerVolumeSpecName "kube-api-access-7z59v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:35.215436 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:35.215405    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b8e3bcdb-a865-428a-8771-6ed558e34369-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:35.215436 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:35.215441    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7z59v\" (UniqueName: \"kubernetes.io/projected/b8e3bcdb-a865-428a-8771-6ed558e34369-kube-api-access-7z59v\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:35.873927 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:35.873891    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"
Oct 30 20:00:35.873927 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:35.873901    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79" event={"ID":"b8e3bcdb-a865-428a-8771-6ed558e34369","Type":"ContainerDied","Data":"036089f7529a252bd63ce7d35745fdfa6ac4849e9d719a81d99e04b2de83b8ce"}
Oct 30 20:00:35.874528 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:35.873970    5704 scope.go:122] "RemoveContainer" containerID="4b902dee181dd11b7ed56adb77e8b0956169eace85c2a69cc298bace43f5756f"
Oct 30 20:00:36.043689 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:36.043653    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"]
Oct 30 20:00:36.088100 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:36.088075    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w2-330025-8595f659cf-mdw79"]
Oct 30 20:00:37.177863 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:37.177829    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b8e3bcdb-a865-428a-8771-6ed558e34369" path="/var/lib/kubelet/pods/b8e3bcdb-a865-428a-8771-6ed558e34369/volumes"
Oct 30 20:00:37.198741 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:37.198702    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:37.630445 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:37.630405    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:37.800352 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:37.800318    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:39.711102 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:39.711060    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:39.921996 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:39.921958    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:41.224348 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.224319    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"]
Oct 30 20:00:41.224777 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.224494    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j" podUID="fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2" containerName="maas-api" containerID="cri-o://32911bc17a18a026aa23b0ba808b5f79580e2347fa75747a317a259dd887fe8d" gracePeriod=30
Oct 30 20:00:41.359354 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.359321    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:41.434762 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.434719    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"]
Oct 30 20:00:41.436275 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.435495    5704 cpu_manager.go:414] "containerMap: removing container" podUID="4015dc19-7986-49c4-a744-2c619f68d81f" containerName="cleanup"
Oct 30 20:00:41.436275 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.435514    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4015dc19-7986-49c4-a744-2c619f68d81f" containerName="cleanup"
Oct 30 20:00:41.436275 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.435548    5704 cpu_manager.go:414] "containerMap: removing container" podUID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerName="istio-proxy"
Oct 30 20:00:41.436275 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.435556    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb4aa4c2-580b-4c9a-a6bf-5e4de536f270" containerName="istio-proxy"
Oct 30 20:00:41.436275 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.435580    5704 cpu_manager.go:414] "containerMap: removing container" podUID="b8e3bcdb-a865-428a-8771-6ed558e34369" containerName="payload-pre-processing"
Oct 30 20:00:41.436275 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.435588    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b8e3bcdb-a865-428a-8771-6ed558e34369" containerName="payload-pre-processing"
Oct 30 20:00:41.436275 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.435617    5704 cpu_manager.go:414] "containerMap: removing container" podUID="4015dc19-7986-49c4-a744-2c619f68d81f" containerName="cleanup"
Oct 30 20:00:41.436275 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.435625    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4015dc19-7986-49c4-a744-2c619f68d81f" containerName="cleanup"
Oct 30 20:00:41.436275 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.435733    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="4015dc19-7986-49c4-a744-2c619f68d81f" containerName="cleanup"
Oct 30 20:00:41.436275 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.435748    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="4015dc19-7986-49c4-a744-2c619f68d81f" containerName="cleanup"
Oct 30 20:00:41.436275 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.435758    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="b8e3bcdb-a865-428a-8771-6ed558e34369" containerName="payload-pre-processing"
Oct 30 20:00:41.439413 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.439390    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.476547 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.476498    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 20:00:41.477020 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.477004    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.477124 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.477102    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.477231 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.477152    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.477231 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.477216    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmsxn\" (UniqueName: \"kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-kube-api-access-rmsxn\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.493498 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.493478    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-b-f77d05\"/\"agc-cred-70369a9b668a\""
Oct 30 20:00:41.498428 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.498406    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"]
Oct 30 20:00:41.578353 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.578323    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-kube-api-access-qbrmm\" (UniqueName: \"kubernetes.io/projected/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-kube-api-access-qbrmm\") pod \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") "
Oct 30 20:00:41.578536 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.578389    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-maas-api-tls\") pod \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") "
Oct 30 20:00:41.578536 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.578444    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-metrics-certs\") pod \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") "
Oct 30 20:00:41.578536 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.578463    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-openshift-service-ca\") pod \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\" (UID: \"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2\") "
Oct 30 20:00:41.578737 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.578587    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.578737 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.578618    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.578737 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.578647    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.578737 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.578715    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmsxn\" (UniqueName: \"kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-kube-api-access-rmsxn\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.578963 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.578873    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-openshift-service-ca" pod "fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2" (UID: "fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:41.579572 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.579550    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.579693 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.579618    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.580718 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.580693    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-metrics-certs" pod "fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2" (UID: "fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:00:41.581046 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.581011    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-maas-api-tls" pod "fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2" (UID: "fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:00:41.581144 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.581018    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-kube-api-access-qbrmm" pod "fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2" (UID: "fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2"). InnerVolumeSpecName "kube-api-access-qbrmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:41.581349 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.581328    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.636104 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.636075    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmsxn\" (UniqueName: \"kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-kube-api-access-rmsxn\") pod \"payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") " pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.679986 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.679953    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:41.679986 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.679979    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:41.680223 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.679992    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbrmm\" (UniqueName: \"kubernetes.io/projected/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-kube-api-access-qbrmm\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:41.680223 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.680007    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:41.728507 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.728435    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:41.754074 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.754045    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:41.907237 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.907199    5704 generic.go:356] "Generic (PLEG): container finished" podID="fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2" containerID="32911bc17a18a026aa23b0ba808b5f79580e2347fa75747a317a259dd887fe8d" exitCode=0
Oct 30 20:00:41.907418 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.907255    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j" event={"ID":"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2","Type":"ContainerDied","Data":"32911bc17a18a026aa23b0ba808b5f79580e2347fa75747a317a259dd887fe8d"}
Oct 30 20:00:41.907418 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.907316    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j" event={"ID":"fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2","Type":"ContainerDied","Data":"d44544eabde03f644071e9924e71b914cdb47be0f5061727386b8d2c5fd3eea9"}
Oct 30 20:00:41.907418 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.907258    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"
Oct 30 20:00:41.907418 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.907336    5704 scope.go:122] "RemoveContainer" containerID="32911bc17a18a026aa23b0ba808b5f79580e2347fa75747a317a259dd887fe8d"
Oct 30 20:00:41.916896 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.916879    5704 scope.go:122] "RemoveContainer" containerID="32911bc17a18a026aa23b0ba808b5f79580e2347fa75747a317a259dd887fe8d"
Oct 30 20:00:41.917167 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:41.917142    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"32911bc17a18a026aa23b0ba808b5f79580e2347fa75747a317a259dd887fe8d\": container with ID starting with 32911bc17a18a026aa23b0ba808b5f79580e2347fa75747a317a259dd887fe8d not found: ID does not exist" containerID="32911bc17a18a026aa23b0ba808b5f79580e2347fa75747a317a259dd887fe8d"
Oct 30 20:00:41.917254 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:41.917169    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"32911bc17a18a026aa23b0ba808b5f79580e2347fa75747a317a259dd887fe8d"} err="failed to get container status \"32911bc17a18a026aa23b0ba808b5f79580e2347fa75747a317a259dd887fe8d\": rpc error: code = NotFound desc = could not find container \"32911bc17a18a026aa23b0ba808b5f79580e2347fa75747a317a259dd887fe8d\": container with ID starting with 32911bc17a18a026aa23b0ba808b5f79580e2347fa75747a317a259dd887fe8d not found: ID does not exist"
Oct 30 20:00:42.061479 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:42.061452    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"]
Oct 30 20:00:42.067317 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:42.067293    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-330025-7c6f9b5ff9-jf86j"]
Oct 30 20:00:42.171281 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:42.171250    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"]
Oct 30 20:00:42.175604 ip-10-0-139-33 kubenswrapper[5704]: W1030 20:00:42.175577    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc56c7254_9b4c_47d9_b26c_1dd3384b04fd.slice/crio-64d0e9dd53e0f3f57a87c472aa9bbc46c8129ea73d8317d24235a68d1ace3359 WatchSource:0}: Error finding container 64d0e9dd53e0f3f57a87c472aa9bbc46c8129ea73d8317d24235a68d1ace3359: Status 404 returned error can't find the container with id 64d0e9dd53e0f3f57a87c472aa9bbc46c8129ea73d8317d24235a68d1ace3359
Oct 30 20:00:42.855829 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:42.855794    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:42.916920 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:42.916886    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz" event={"ID":"c56c7254-9b4c-47d9-b26c-1dd3384b04fd","Type":"ContainerStarted","Data":"9bfe119147fc0e75df68255d49fce8a9166d6e90c4ef68c3be167b8fa96df73a"}
Oct 30 20:00:42.917157 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:42.916942    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz" event={"ID":"c56c7254-9b4c-47d9-b26c-1dd3384b04fd","Type":"ContainerStarted","Data":"64d0e9dd53e0f3f57a87c472aa9bbc46c8129ea73d8317d24235a68d1ace3359"}
Oct 30 20:00:42.917157 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:42.916966    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:43.040653 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:43.040606    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz" podStartSLOduration=2.040593441 podStartE2EDuration="2.040593441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:43.039283291 +0000 UTC m=+4415.985451195" watchObservedRunningTime="2026-10-30 20:00:43.040593441 +0000 UTC m=+4415.986761336"
Oct 30 20:00:43.075277 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:43.075239    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:43.178628 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:43.178553    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2" path="/var/lib/kubelet/pods/fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2/volumes"
Oct 30 20:00:43.257867 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:43.257837    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:43.783298 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:43.783271    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:44.036388 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.036298    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:44.540304 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.540271    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"]
Oct 30 20:00:44.540816 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.540798    5704 cpu_manager.go:414] "containerMap: removing container" podUID="fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2" containerName="maas-api"
Oct 30 20:00:44.540918 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.540818    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2" containerName="maas-api"
Oct 30 20:00:44.540918 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.540843    5704 cpu_manager.go:414] "containerMap: removing container" podUID="4015dc19-7986-49c4-a744-2c619f68d81f" containerName="cleanup"
Oct 30 20:00:44.540918 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.540859    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4015dc19-7986-49c4-a744-2c619f68d81f" containerName="cleanup"
Oct 30 20:00:44.541137 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.540979    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="4015dc19-7986-49c4-a744-2c619f68d81f" containerName="cleanup"
Oct 30 20:00:44.541137 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.541001    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc2381bc-b8b5-4d55-8c3c-6df0b2c77fe2" containerName="maas-api"
Oct 30 20:00:44.544984 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.544963    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:44.598097 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.598070    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-d50b07\"/\"kube-root-ca.crt\""
Oct 30 20:00:44.615916 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.615886    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") " pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:44.616071 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.615958    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrpk4\" (UniqueName: \"kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-kube-api-access-xrpk4\") pod \"payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") " pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:44.616071 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.615994    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") " pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:44.616218 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.616092    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") " pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:44.640486 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.640446    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-d50b07\"/\"openshift-service-ca.crt\""
Oct 30 20:00:44.640486 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.640446    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-d50b07\"/\"agc-cred-b25d78e7332d\""
Oct 30 20:00:44.640799 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.640553    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-extmod-a-d50b07\"/\"payload-processing-external-model-e2e--f418ff59-dockercfg-jm52f\""
Oct 30 20:00:44.640799 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.640552    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-d50b07\"/\"payload-processing-external-model-plugins-e2e-extmod-a-d50b07\""
Oct 30 20:00:44.641080 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.641027    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-extmod-a-d50b07\"/\"routing-overlay\""
Oct 30 20:00:44.648344 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.648317    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"]
Oct 30 20:00:44.717290 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.717263    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") " pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:44.717429 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.717344    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") " pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:44.717429 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.717374    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrpk4\" (UniqueName: \"kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-kube-api-access-xrpk4\") pod \"payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") " pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:44.717429 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.717395    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") " pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:44.718203 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.718169    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-routing-overlay-volume\") pod \"payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") " pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:44.718328 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.718209    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-plugins-config-volume\") pod \"payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") " pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:44.719751 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.719726    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials\") pod \"payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") " pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:44.776435 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.776406    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrpk4\" (UniqueName: \"kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-kube-api-access-xrpk4\") pod \"payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") " pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:44.858614 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:44.858552    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:45.234755 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:45.234678    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"]
Oct 30 20:00:45.240317 ip-10-0-139-33 kubenswrapper[5704]: W1030 20:00:45.240286    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedd8b872_d7f3_4cc3_b16c_59b29329b524.slice/crio-573a133ad01027f31e1777291f423acf29308645f9a03fdb398f189b40984670 WatchSource:0}: Error finding container 573a133ad01027f31e1777291f423acf29308645f9a03fdb398f189b40984670: Status 404 returned error can't find the container with id 573a133ad01027f31e1777291f423acf29308645f9a03fdb398f189b40984670
Oct 30 20:00:45.641598 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:45.641564    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:45.908456 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:45.908352    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:45.934259 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:45.934233    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4" event={"ID":"edd8b872-d7f3-4cc3-b16c-59b29329b524","Type":"ContainerStarted","Data":"da64deeeff02111afe913da241cabc1dc82cd4e03dc3c085234f657bdd6a367e"}
Oct 30 20:00:45.934399 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:45.934305    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4" event={"ID":"edd8b872-d7f3-4cc3-b16c-59b29329b524","Type":"ContainerStarted","Data":"573a133ad01027f31e1777291f423acf29308645f9a03fdb398f189b40984670"}
Oct 30 20:00:45.934399 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:45.934327    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:46.034401 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:46.034369    5704 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-d50b07/agc-cred-b25d78e7332d: secret "agc-cred-b25d78e7332d" not found
Oct 30 20:00:46.034401 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:46.034394    5704 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4: secret "agc-cred-b25d78e7332d" not found
Oct 30 20:00:46.034620 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:46.034443    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials podName:edd8b872-d7f3-4cc3-b16c-59b29329b524 nodeName:}" failed. No retries permitted until 2026-10-30 20:00:46.53442947 +0000 UTC m=+4419.480597365 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4" (UID: "edd8b872-d7f3-4cc3-b16c-59b29329b524") : secret "agc-cred-b25d78e7332d" not found
Oct 30 20:00:46.048524 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:46.048485    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4" podStartSLOduration=2.048470758 podStartE2EDuration="2.048470758s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:00:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:00:46.046935682 +0000 UTC m=+4418.993103585" watchObservedRunningTime="2026-10-30 20:00:46.048470758 +0000 UTC m=+4418.994638662"
Oct 30 20:00:46.473743 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:46.473705    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"]
Oct 30 20:00:46.538611 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:46.538582    5704 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-d50b07/agc-cred-b25d78e7332d: secret "agc-cred-b25d78e7332d" not found
Oct 30 20:00:46.538611 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:46.538604    5704 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4: secret "agc-cred-b25d78e7332d" not found
Oct 30 20:00:46.538805 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:46.538654    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials podName:edd8b872-d7f3-4cc3-b16c-59b29329b524 nodeName:}" failed. No retries permitted until 2026-10-30 20:00:47.538641517 +0000 UTC m=+4420.484809398 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4" (UID: "edd8b872-d7f3-4cc3-b16c-59b29329b524") : secret "agc-cred-b25d78e7332d" not found
Oct 30 20:00:47.548959 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:47.548926    5704 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-d50b07/agc-cred-b25d78e7332d: secret "agc-cred-b25d78e7332d" not found
Oct 30 20:00:47.548959 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:47.548951    5704 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4: secret "agc-cred-b25d78e7332d" not found
Oct 30 20:00:47.549441 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:47.548998    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials podName:edd8b872-d7f3-4cc3-b16c-59b29329b524 nodeName:}" failed. No retries permitted until 2026-10-30 20:00:49.548985808 +0000 UTC m=+4422.495153689 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4" (UID: "edd8b872-d7f3-4cc3-b16c-59b29329b524") : secret "agc-cred-b25d78e7332d" not found
Oct 30 20:00:47.974172 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:47.974059    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"]
Oct 30 20:00:48.156879 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:48.156826    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:48.340194 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:48.340161    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:48.813325 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:48.813283    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:49.001217 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:49.001179    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:49.568966 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:49.568938    5704 projected.go:266] Couldn't get secret ai-tenant-e2e-extmod-a-d50b07/agc-cred-b25d78e7332d: secret "agc-cred-b25d78e7332d" not found
Oct 30 20:00:49.568966 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:49.568962    5704 projected.go:196] Error preparing data for projected volume provider-credentials for pod ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4: secret "agc-cred-b25d78e7332d" not found
Oct 30 20:00:49.569226 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:49.569011    5704 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials podName:edd8b872-d7f3-4cc3-b16c-59b29329b524 nodeName:}" failed. No retries permitted until 2026-10-30 20:00:53.568996216 +0000 UTC m=+4426.515164100 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "provider-credentials" (UniqueName: "kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials") pod "payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4" (UID: "edd8b872-d7f3-4cc3-b16c-59b29329b524") : secret "agc-cred-b25d78e7332d" not found
Oct 30 20:00:49.699294 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:49.699263    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:49.927138 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:49.927047    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:50.190712 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:50.190631    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:50.369726 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:50.369696    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:51.008425 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.008389    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:51.189398 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.189366    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:51.311363 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.311292    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"]
Oct 30 20:00:51.311522 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.311497    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp" podUID="d4171f5d-d080-40d0-8a48-f592ba37f8d6" containerName="istio-proxy" containerID="cri-o://efe220f8c7a9f76a58881e0f10e11b7868c279c5041455ac57c531d9cee9cf69" gracePeriod=30
Oct 30 20:00:51.400547 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.400505    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:51.474786 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.474748    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz" podUID="c56c7254-9b4c-47d9-b26c-1dd3384b04fd" containerName="payload-processing" containerID="cri-o://9bfe119147fc0e75df68255d49fce8a9166d6e90c4ef68c3be167b8fa96df73a" gracePeriod=25
Oct 30 20:00:51.725583 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.725560    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:51.791892 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.791870    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-plugins-config-volume\") pod \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") "
Oct 30 20:00:51.792051 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.791913    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-provider-credentials\" (UniqueName: \"kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-provider-credentials\") pod \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") "
Oct 30 20:00:51.792051 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.791953    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-routing-overlay-volume\") pod \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") "
Oct 30 20:00:51.792051 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.791983    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-kube-api-access-rmsxn\" (UniqueName: \"kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-kube-api-access-rmsxn\") pod \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\" (UID: \"c56c7254-9b4c-47d9-b26c-1dd3384b04fd\") "
Oct 30 20:00:51.792342 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.792319    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-plugins-config-volume" pod "c56c7254-9b4c-47d9-b26c-1dd3384b04fd" (UID: "c56c7254-9b4c-47d9-b26c-1dd3384b04fd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:51.792421 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.792338    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-routing-overlay-volume" pod "c56c7254-9b4c-47d9-b26c-1dd3384b04fd" (UID: "c56c7254-9b4c-47d9-b26c-1dd3384b04fd"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:51.792488 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.792471    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:51.792525 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.792490    5704 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-routing-overlay-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:51.794234 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.794206    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-provider-credentials" pod "c56c7254-9b4c-47d9-b26c-1dd3384b04fd" (UID: "c56c7254-9b4c-47d9-b26c-1dd3384b04fd"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:51.794329 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.794248    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-kube-api-access-rmsxn" pod "c56c7254-9b4c-47d9-b26c-1dd3384b04fd" (UID: "c56c7254-9b4c-47d9-b26c-1dd3384b04fd"). InnerVolumeSpecName "kube-api-access-rmsxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:51.893202 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.893136    5704 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-provider-credentials\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:51.893202 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.893160    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmsxn\" (UniqueName: \"kubernetes.io/projected/c56c7254-9b4c-47d9-b26c-1dd3384b04fd-kube-api-access-rmsxn\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:51.969376 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.969346    5704 generic.go:356] "Generic (PLEG): container finished" podID="c56c7254-9b4c-47d9-b26c-1dd3384b04fd" containerID="9bfe119147fc0e75df68255d49fce8a9166d6e90c4ef68c3be167b8fa96df73a" exitCode=0
Oct 30 20:00:51.969550 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.969405    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"
Oct 30 20:00:51.969550 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.969438    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz" event={"ID":"c56c7254-9b4c-47d9-b26c-1dd3384b04fd","Type":"ContainerDied","Data":"9bfe119147fc0e75df68255d49fce8a9166d6e90c4ef68c3be167b8fa96df73a"}
Oct 30 20:00:51.969550 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.969515    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz" event={"ID":"c56c7254-9b4c-47d9-b26c-1dd3384b04fd","Type":"ContainerDied","Data":"64d0e9dd53e0f3f57a87c472aa9bbc46c8129ea73d8317d24235a68d1ace3359"}
Oct 30 20:00:51.969550 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.969538    5704 scope.go:122] "RemoveContainer" containerID="9bfe119147fc0e75df68255d49fce8a9166d6e90c4ef68c3be167b8fa96df73a"
Oct 30 20:00:51.978651 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.978634    5704 scope.go:122] "RemoveContainer" containerID="9bfe119147fc0e75df68255d49fce8a9166d6e90c4ef68c3be167b8fa96df73a"
Oct 30 20:00:51.978907 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:51.978886    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bfe119147fc0e75df68255d49fce8a9166d6e90c4ef68c3be167b8fa96df73a\": container with ID starting with 9bfe119147fc0e75df68255d49fce8a9166d6e90c4ef68c3be167b8fa96df73a not found: ID does not exist" containerID="9bfe119147fc0e75df68255d49fce8a9166d6e90c4ef68c3be167b8fa96df73a"
Oct 30 20:00:51.978965 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:51.978916    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bfe119147fc0e75df68255d49fce8a9166d6e90c4ef68c3be167b8fa96df73a"} err="failed to get container status \"9bfe119147fc0e75df68255d49fce8a9166d6e90c4ef68c3be167b8fa96df73a\": rpc error: code = NotFound desc = could not find container \"9bfe119147fc0e75df68255d49fce8a9166d6e90c4ef68c3be167b8fa96df73a\": container with ID starting with 9bfe119147fc0e75df68255d49fce8a9166d6e90c4ef68c3be167b8fa96df73a not found: ID does not exist"
Oct 30 20:00:52.121962 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:52.121935    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"]
Oct 30 20:00:52.137264 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:52.137235    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-55c65cgmz"]
Oct 30 20:00:52.901509 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:52.901474    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:52.974362 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:52.974325    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4" podUID="edd8b872-d7f3-4cc3-b16c-59b29329b524" containerName="payload-processing" containerID="cri-o://da64deeeff02111afe913da241cabc1dc82cd4e03dc3c085234f657bdd6a367e" gracePeriod=25
Oct 30 20:00:53.180264 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.180193    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c56c7254-9b4c-47d9-b26c-1dd3384b04fd" path="/var/lib/kubelet/pods/c56c7254-9b4c-47d9-b26c-1dd3384b04fd/volumes"
Oct 30 20:00:53.226498 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.226472    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:53.307471 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.307450    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials\" (UniqueName: \"kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials\") pod \"edd8b872-d7f3-4cc3-b16c-59b29329b524\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") "
Oct 30 20:00:53.307592 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.307480    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-routing-overlay-volume\") pod \"edd8b872-d7f3-4cc3-b16c-59b29329b524\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") "
Oct 30 20:00:53.307592 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.307558    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-plugins-config-volume\") pod \"edd8b872-d7f3-4cc3-b16c-59b29329b524\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") "
Oct 30 20:00:53.307678 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.307610    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-kube-api-access-xrpk4\" (UniqueName: \"kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-kube-api-access-xrpk4\") pod \"edd8b872-d7f3-4cc3-b16c-59b29329b524\" (UID: \"edd8b872-d7f3-4cc3-b16c-59b29329b524\") "
Oct 30 20:00:53.307867 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.307845    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-routing-overlay-volume" pod "edd8b872-d7f3-4cc3-b16c-59b29329b524" (UID: "edd8b872-d7f3-4cc3-b16c-59b29329b524"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:53.307951 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.307929    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-plugins-config-volume" pod "edd8b872-d7f3-4cc3-b16c-59b29329b524" (UID: "edd8b872-d7f3-4cc3-b16c-59b29329b524"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:53.308024 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.307995    5704 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-routing-overlay-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:53.308024 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.308007    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/edd8b872-d7f3-4cc3-b16c-59b29329b524-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:53.309613 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.309594    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials" pod "edd8b872-d7f3-4cc3-b16c-59b29329b524" (UID: "edd8b872-d7f3-4cc3-b16c-59b29329b524"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:53.309706 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.309666    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-kube-api-access-xrpk4" pod "edd8b872-d7f3-4cc3-b16c-59b29329b524" (UID: "edd8b872-d7f3-4cc3-b16c-59b29329b524"). InnerVolumeSpecName "kube-api-access-xrpk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:53.409447 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.409415    5704 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-provider-credentials\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:53.409447 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.409441    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xrpk4\" (UniqueName: \"kubernetes.io/projected/edd8b872-d7f3-4cc3-b16c-59b29329b524-kube-api-access-xrpk4\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:53.980819 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.980790    5704 generic.go:356] "Generic (PLEG): container finished" podID="edd8b872-d7f3-4cc3-b16c-59b29329b524" containerID="da64deeeff02111afe913da241cabc1dc82cd4e03dc3c085234f657bdd6a367e" exitCode=0
Oct 30 20:00:53.980989 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.980828    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4" event={"ID":"edd8b872-d7f3-4cc3-b16c-59b29329b524","Type":"ContainerDied","Data":"da64deeeff02111afe913da241cabc1dc82cd4e03dc3c085234f657bdd6a367e"}
Oct 30 20:00:53.980989 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.980831    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"
Oct 30 20:00:53.980989 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.980878    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4" event={"ID":"edd8b872-d7f3-4cc3-b16c-59b29329b524","Type":"ContainerDied","Data":"573a133ad01027f31e1777291f423acf29308645f9a03fdb398f189b40984670"}
Oct 30 20:00:53.980989 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.980900    5704 scope.go:122] "RemoveContainer" containerID="da64deeeff02111afe913da241cabc1dc82cd4e03dc3c085234f657bdd6a367e"
Oct 30 20:00:53.990077 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.990056    5704 scope.go:122] "RemoveContainer" containerID="da64deeeff02111afe913da241cabc1dc82cd4e03dc3c085234f657bdd6a367e"
Oct 30 20:00:53.990335 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:53.990315    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da64deeeff02111afe913da241cabc1dc82cd4e03dc3c085234f657bdd6a367e\": container with ID starting with da64deeeff02111afe913da241cabc1dc82cd4e03dc3c085234f657bdd6a367e not found: ID does not exist" containerID="da64deeeff02111afe913da241cabc1dc82cd4e03dc3c085234f657bdd6a367e"
Oct 30 20:00:53.990397 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:53.990344    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da64deeeff02111afe913da241cabc1dc82cd4e03dc3c085234f657bdd6a367e"} err="failed to get container status \"da64deeeff02111afe913da241cabc1dc82cd4e03dc3c085234f657bdd6a367e\": rpc error: code = NotFound desc = could not find container \"da64deeeff02111afe913da241cabc1dc82cd4e03dc3c085234f657bdd6a367e\": container with ID starting with da64deeeff02111afe913da241cabc1dc82cd4e03dc3c085234f657bdd6a367e not found: ID does not exist"
Oct 30 20:00:54.144063 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:54.143969    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"]
Oct 30 20:00:54.189802 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:54.189779    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-a-d50b07/payload-processing-external-model-e2e-extmod-a-d50b07-7d99bgjs4"]
Oct 30 20:00:55.178415 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:55.178382    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="edd8b872-d7f3-4cc3-b16c-59b29329b524" path="/var/lib/kubelet/pods/edd8b872-d7f3-4cc3-b16c-59b29329b524/volumes"
Oct 30 20:00:56.555101 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.555076    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 20:00:56.640833 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.640776    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istiod-ca-cert\") pod \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") "
Oct 30 20:00:56.640833 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.640827    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-envoy\") pod \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") "
Oct 30 20:00:56.640984 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.640886    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-podinfo\") pod \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") "
Oct 30 20:00:56.640984 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.640928    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-data\") pod \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") "
Oct 30 20:00:56.641098 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641027    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-credential-socket\") pod \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") "
Oct 30 20:00:56.641159 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641095    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-kube-api-access-z594b\" (UniqueName: \"kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-kube-api-access-z594b\") pod \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") "
Oct 30 20:00:56.641159 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641145    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-socket\") pod \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") "
Oct 30 20:00:56.641275 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641187    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-token\" (UniqueName: \"kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-token\") pod \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") "
Oct 30 20:00:56.641348 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641189    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istiod-ca-cert" pod "d4171f5d-d080-40d0-8a48-f592ba37f8d6" (UID: "d4171f5d-d080-40d0-8a48-f592ba37f8d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:00:56.641348 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641219    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-data" pod "d4171f5d-d080-40d0-8a48-f592ba37f8d6" (UID: "d4171f5d-d080-40d0-8a48-f592ba37f8d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:56.641348 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641271    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-credential-socket" pod "d4171f5d-d080-40d0-8a48-f592ba37f8d6" (UID: "d4171f5d-d080-40d0-8a48-f592ba37f8d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:56.641348 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641226    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-certs\") pod \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\" (UID: \"d4171f5d-d080-40d0-8a48-f592ba37f8d6\") "
Oct 30 20:00:56.641567 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641461    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-certs" pod "d4171f5d-d080-40d0-8a48-f592ba37f8d6" (UID: "d4171f5d-d080-40d0-8a48-f592ba37f8d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:56.641567 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641475    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-socket" pod "d4171f5d-d080-40d0-8a48-f592ba37f8d6" (UID: "d4171f5d-d080-40d0-8a48-f592ba37f8d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:56.641758 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641737    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:56.641848 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641764    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:56.641848 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641780    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:56.641848 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641793    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:56.641848 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.641807    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:56.643048 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.643010    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-envoy" pod "d4171f5d-d080-40d0-8a48-f592ba37f8d6" (UID: "d4171f5d-d080-40d0-8a48-f592ba37f8d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:00:56.643197 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.643170    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-kube-api-access-z594b" pod "d4171f5d-d080-40d0-8a48-f592ba37f8d6" (UID: "d4171f5d-d080-40d0-8a48-f592ba37f8d6"). InnerVolumeSpecName "kube-api-access-z594b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:56.643356 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.643335    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-podinfo" pod "d4171f5d-d080-40d0-8a48-f592ba37f8d6" (UID: "d4171f5d-d080-40d0-8a48-f592ba37f8d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:00:56.643541 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.643523    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-token" pod "d4171f5d-d080-40d0-8a48-f592ba37f8d6" (UID: "d4171f5d-d080-40d0-8a48-f592ba37f8d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:00:56.742767 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.742739    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:56.742767 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.742762    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:56.742939 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.742773    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4171f5d-d080-40d0-8a48-f592ba37f8d6-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:56.742939 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.742782    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z594b\" (UniqueName: \"kubernetes.io/projected/d4171f5d-d080-40d0-8a48-f592ba37f8d6-kube-api-access-z594b\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:00:56.998687 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.998654    5704 generic.go:356] "Generic (PLEG): container finished" podID="d4171f5d-d080-40d0-8a48-f592ba37f8d6" containerID="efe220f8c7a9f76a58881e0f10e11b7868c279c5041455ac57c531d9cee9cf69" exitCode=0
Oct 30 20:00:56.998852 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.998711    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"
Oct 30 20:00:56.998852 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.998737    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp" event={"ID":"d4171f5d-d080-40d0-8a48-f592ba37f8d6","Type":"ContainerDied","Data":"efe220f8c7a9f76a58881e0f10e11b7868c279c5041455ac57c531d9cee9cf69"}
Oct 30 20:00:56.998852 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.998807    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp" event={"ID":"d4171f5d-d080-40d0-8a48-f592ba37f8d6","Type":"ContainerDied","Data":"165e1a86b0bf4fe048fe0339c680d371e686ed6675c9d9911fedc015a2773670"}
Oct 30 20:00:56.998852 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:56.998829    5704 scope.go:122] "RemoveContainer" containerID="efe220f8c7a9f76a58881e0f10e11b7868c279c5041455ac57c531d9cee9cf69"
Oct 30 20:00:57.008435 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:57.008412    5704 scope.go:122] "RemoveContainer" containerID="efe220f8c7a9f76a58881e0f10e11b7868c279c5041455ac57c531d9cee9cf69"
Oct 30 20:00:57.008687 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:00:57.008668    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efe220f8c7a9f76a58881e0f10e11b7868c279c5041455ac57c531d9cee9cf69\": container with ID starting with efe220f8c7a9f76a58881e0f10e11b7868c279c5041455ac57c531d9cee9cf69 not found: ID does not exist" containerID="efe220f8c7a9f76a58881e0f10e11b7868c279c5041455ac57c531d9cee9cf69"
Oct 30 20:00:57.008760 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:57.008700    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efe220f8c7a9f76a58881e0f10e11b7868c279c5041455ac57c531d9cee9cf69"} err="failed to get container status \"efe220f8c7a9f76a58881e0f10e11b7868c279c5041455ac57c531d9cee9cf69\": rpc error: code = NotFound desc = could not find container \"efe220f8c7a9f76a58881e0f10e11b7868c279c5041455ac57c531d9cee9cf69\": container with ID starting with efe220f8c7a9f76a58881e0f10e11b7868c279c5041455ac57c531d9cee9cf69 not found: ID does not exist"
Oct 30 20:00:57.153434 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:57.153410    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"]
Oct 30 20:00:57.157066 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:57.157019    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-330025-openshift-default-5c47dd8bc5-vlcqp"]
Oct 30 20:00:57.178792 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:57.178770    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d4171f5d-d080-40d0-8a48-f592ba37f8d6" path="/var/lib/kubelet/pods/d4171f5d-d080-40d0-8a48-f592ba37f8d6/volumes"
Oct 30 20:00:57.687445 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:57.687411    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:00:59.406848 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:00:59.406810    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 20:01:02.378147 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:02.378110    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:03.383283 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:01:03.383225    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 20:01:04.138455 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:04.138413    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:04.172799 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:04.172761    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"]
Oct 30 20:01:04.324277 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:04.324239    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:04.508336 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:04.508305    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:04.947159 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:04.947066    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:06.973278 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:06.973242    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:09.173964 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.173928    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6" podUID="6db8a91c-6926-4df9-8c74-453363556742" containerName="payload-processing" containerID="cri-o://8c4ce6cec246120021908732acffcfbf28ad26254abc89e0f8b112c21f6d11b7" gracePeriod=25
Oct 30 20:01:09.417271 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.417247    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 20:01:09.461737 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.461662    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-routing-overlay-volume\") pod \"6db8a91c-6926-4df9-8c74-453363556742\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") "
Oct 30 20:01:09.461737 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.461719    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-plugins-config-volume\") pod \"6db8a91c-6926-4df9-8c74-453363556742\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") "
Oct 30 20:01:09.461966 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.461796    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-provider-credentials\" (UniqueName: \"kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-provider-credentials\") pod \"6db8a91c-6926-4df9-8c74-453363556742\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") "
Oct 30 20:01:09.461966 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.461824    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-kube-api-access-spwln\" (UniqueName: \"kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-kube-api-access-spwln\") pod \"6db8a91c-6926-4df9-8c74-453363556742\" (UID: \"6db8a91c-6926-4df9-8c74-453363556742\") "
Oct 30 20:01:09.462126 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.462100    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-routing-overlay-volume" pod "6db8a91c-6926-4df9-8c74-453363556742" (UID: "6db8a91c-6926-4df9-8c74-453363556742"). InnerVolumeSpecName "routing-overlay-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:01:09.462189 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.462164    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-plugins-config-volume" pod "6db8a91c-6926-4df9-8c74-453363556742" (UID: "6db8a91c-6926-4df9-8c74-453363556742"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:01:09.463766 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.463742    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-provider-credentials" pod "6db8a91c-6926-4df9-8c74-453363556742" (UID: "6db8a91c-6926-4df9-8c74-453363556742"). InnerVolumeSpecName "provider-credentials". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:01:09.463949 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.463927    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-kube-api-access-spwln" pod "6db8a91c-6926-4df9-8c74-453363556742" (UID: "6db8a91c-6926-4df9-8c74-453363556742"). InnerVolumeSpecName "kube-api-access-spwln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:01:09.562501 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.562464    5704 reconciler_common.go:299] "Volume detached for volume \"routing-overlay-volume\" (UniqueName: \"kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-routing-overlay-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:01:09.562501 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.562496    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6db8a91c-6926-4df9-8c74-453363556742-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:01:09.562501 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.562508    5704 reconciler_common.go:299] "Volume detached for volume \"provider-credentials\" (UniqueName: \"kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-provider-credentials\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:01:09.562733 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:09.562521    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spwln\" (UniqueName: \"kubernetes.io/projected/6db8a91c-6926-4df9-8c74-453363556742-kube-api-access-spwln\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:01:10.068320 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:10.068287    5704 generic.go:356] "Generic (PLEG): container finished" podID="6db8a91c-6926-4df9-8c74-453363556742" containerID="8c4ce6cec246120021908732acffcfbf28ad26254abc89e0f8b112c21f6d11b7" exitCode=0
Oct 30 20:01:10.068534 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:10.068342    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6" event={"ID":"6db8a91c-6926-4df9-8c74-453363556742","Type":"ContainerDied","Data":"8c4ce6cec246120021908732acffcfbf28ad26254abc89e0f8b112c21f6d11b7"}
Oct 30 20:01:10.068534 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:10.068347    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"
Oct 30 20:01:10.068534 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:10.068387    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6" event={"ID":"6db8a91c-6926-4df9-8c74-453363556742","Type":"ContainerDied","Data":"6836050ee42e4e30d4752f744ae4877111b3095e8814790c138a825f9dbc9be7"}
Oct 30 20:01:10.068534 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:10.068409    5704 scope.go:122] "RemoveContainer" containerID="8c4ce6cec246120021908732acffcfbf28ad26254abc89e0f8b112c21f6d11b7"
Oct 30 20:01:10.078219 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:10.078202    5704 scope.go:122] "RemoveContainer" containerID="8c4ce6cec246120021908732acffcfbf28ad26254abc89e0f8b112c21f6d11b7"
Oct 30 20:01:10.078465 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:01:10.078446    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c4ce6cec246120021908732acffcfbf28ad26254abc89e0f8b112c21f6d11b7\": container with ID starting with 8c4ce6cec246120021908732acffcfbf28ad26254abc89e0f8b112c21f6d11b7 not found: ID does not exist" containerID="8c4ce6cec246120021908732acffcfbf28ad26254abc89e0f8b112c21f6d11b7"
Oct 30 20:01:10.078535 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:10.078477    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c4ce6cec246120021908732acffcfbf28ad26254abc89e0f8b112c21f6d11b7"} err="failed to get container status \"8c4ce6cec246120021908732acffcfbf28ad26254abc89e0f8b112c21f6d11b7\": rpc error: code = NotFound desc = could not find container \"8c4ce6cec246120021908732acffcfbf28ad26254abc89e0f8b112c21f6d11b7\": container with ID starting with 8c4ce6cec246120021908732acffcfbf28ad26254abc89e0f8b112c21f6d11b7 not found: ID does not exist"
Oct 30 20:01:10.221278 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:10.221250    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"]
Oct 30 20:01:10.266745 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:10.266719    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-extmod-b-f77d05/payload-processing-external-model-e2e-extmod-b-f77d05-7cc5cpcb6"]
Oct 30 20:01:11.178295 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:11.178262    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6db8a91c-6926-4df9-8c74-453363556742" path="/var/lib/kubelet/pods/6db8a91c-6926-4df9-8c74-453363556742/volumes"
Oct 30 20:01:11.351837 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:11.351799    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:11.536568 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:11.536534    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:11.713914 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:11.713882    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:16.268193 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:16.268157    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:16.532822 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:16.532747    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 20:01:18.280929 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:18.280894    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:18.486465 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:18.486435    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:19.280252 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:19.280224    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:21.182547 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:21.182514    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:26.187942 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:26.187909    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:30.521238 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:30.521197    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:30.755960 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:30.755920    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:30.933886 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:30.933792    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:33.383582 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:01:33.383548    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 20:01:35.366960 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:35.366925    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:37.208441 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:37.208408    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:40.076951 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:40.076917    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:44.062567 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:44.062531    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:45.582412 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:45.582379    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:45.837924 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:45.837837    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:46.520918 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:46.520881    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:49.443186 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:49.443157    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 20:01:50.573481 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:50.573441    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:52.736398 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:52.736367    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:52.983002 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:52.982969    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:53.179357 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:53.179324    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:01:55.416874 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:01:55.416831    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:00.181291 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:00.181257    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:03.382863 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:02:03.382824    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 20:02:04.778426 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:04.778367    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:05.343012 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:05.342981    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"]
Oct 30 20:02:05.443880 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:05.443852    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"]
Oct 30 20:02:05.729383 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:05.729291    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:05.923619 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:05.923579    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:09.465981 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:09.465952    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:10.343777 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:10.343723    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt" podUID="7d4049f4-af8d-407e-b344-b8730a24b4e2" containerName="payload-processing" containerID="cri-o://3817b8a5739f04748945a8c737ab43ef2a715e3c67401e8a5463739e5581fa5d" gracePeriod=25
Oct 30 20:02:10.444315 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:10.444279    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp" podUID="53821e33-7446-47cd-afd1-7489845cda73" containerName="payload-pre-processing" containerID="cri-o://70ddd254418a7bc7001052793c151c32b171e82af6c9b34df8cc403928dc9440" gracePeriod=25
Oct 30 20:02:13.484900 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:13.484862    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:14.993457 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:14.993415    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:15.217937 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:15.217903    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:15.436801 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:15.436769    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:15.963025 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:15.962992    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:15.973735 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:15.973701    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp" podUID="53821e33-7446-47cd-afd1-7489845cda73" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:02:15.973856 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:15.973702    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt" podUID="7d4049f4-af8d-407e-b344-b8730a24b4e2" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:02:18.741324 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:18.741252    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 20:02:19.600316 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:19.600277    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:20.071240 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:20.071210    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:22.827883 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:22.827842    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:23.050416 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:23.050379    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:23.277708 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:23.277675    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:23.732966 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:23.732896    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:24.771876 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:24.771840    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:25.974346 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:25.974306    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp" podUID="53821e33-7446-47cd-afd1-7489845cda73" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:02:25.974722 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:25.974317    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt" podUID="7d4049f4-af8d-407e-b344-b8730a24b4e2" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")"
Oct 30 20:02:28.629701 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:28.629671    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"]
Oct 30 20:02:28.808773 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:28.808737    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:29.878093 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:29.878027    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:30.593827 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.593805    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"
Oct 30 20:02:30.691114 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.691089    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"
Oct 30 20:02:30.691313 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.691294    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d4049f4-af8d-407e-b344-b8730a24b4e2-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d4049f4-af8d-407e-b344-b8730a24b4e2-plugins-config-volume\") pod \"7d4049f4-af8d-407e-b344-b8730a24b4e2\" (UID: \"7d4049f4-af8d-407e-b344-b8730a24b4e2\") "
Oct 30 20:02:30.691394 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.691339    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d4049f4-af8d-407e-b344-b8730a24b4e2-kube-api-access-vpnk5\" (UniqueName: \"kubernetes.io/projected/7d4049f4-af8d-407e-b344-b8730a24b4e2-kube-api-access-vpnk5\") pod \"7d4049f4-af8d-407e-b344-b8730a24b4e2\" (UID: \"7d4049f4-af8d-407e-b344-b8730a24b4e2\") "
Oct 30 20:02:30.691634 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.691614    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d4049f4-af8d-407e-b344-b8730a24b4e2-plugins-config-volume" pod "7d4049f4-af8d-407e-b344-b8730a24b4e2" (UID: "7d4049f4-af8d-407e-b344-b8730a24b4e2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:02:30.693302 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.693284    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d4049f4-af8d-407e-b344-b8730a24b4e2-kube-api-access-vpnk5" pod "7d4049f4-af8d-407e-b344-b8730a24b4e2" (UID: "7d4049f4-af8d-407e-b344-b8730a24b4e2"). InnerVolumeSpecName "kube-api-access-vpnk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:02:30.792516 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.792494    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/53821e33-7446-47cd-afd1-7489845cda73-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53821e33-7446-47cd-afd1-7489845cda73-plugins-config-volume\") pod \"53821e33-7446-47cd-afd1-7489845cda73\" (UID: \"53821e33-7446-47cd-afd1-7489845cda73\") "
Oct 30 20:02:30.792686 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.792551    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/53821e33-7446-47cd-afd1-7489845cda73-kube-api-access-55j4d\" (UniqueName: \"kubernetes.io/projected/53821e33-7446-47cd-afd1-7489845cda73-kube-api-access-55j4d\") pod \"53821e33-7446-47cd-afd1-7489845cda73\" (UID: \"53821e33-7446-47cd-afd1-7489845cda73\") "
Oct 30 20:02:30.792811 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.792799    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d4049f4-af8d-407e-b344-b8730a24b4e2-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:30.792887 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.792813    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpnk5\" (UniqueName: \"kubernetes.io/projected/7d4049f4-af8d-407e-b344-b8730a24b4e2-kube-api-access-vpnk5\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:30.792951 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.792909    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53821e33-7446-47cd-afd1-7489845cda73-plugins-config-volume" pod "53821e33-7446-47cd-afd1-7489845cda73" (UID: "53821e33-7446-47cd-afd1-7489845cda73"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:02:30.794507 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.794488    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/53821e33-7446-47cd-afd1-7489845cda73-kube-api-access-55j4d" pod "53821e33-7446-47cd-afd1-7489845cda73" (UID: "53821e33-7446-47cd-afd1-7489845cda73"). InnerVolumeSpecName "kube-api-access-55j4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:02:30.893436 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.893351    5704 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/53821e33-7446-47cd-afd1-7489845cda73-plugins-config-volume\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:30.893436 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:30.893379    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55j4d\" (UniqueName: \"kubernetes.io/projected/53821e33-7446-47cd-afd1-7489845cda73-kube-api-access-55j4d\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:31.459538 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.459461    5704 generic.go:356] "Generic (PLEG): container finished" podID="53821e33-7446-47cd-afd1-7489845cda73" containerID="70ddd254418a7bc7001052793c151c32b171e82af6c9b34df8cc403928dc9440" exitCode=0
Oct 30 20:02:31.459538 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.459511    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"
Oct 30 20:02:31.459813 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.459551    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp" event={"ID":"53821e33-7446-47cd-afd1-7489845cda73","Type":"ContainerDied","Data":"70ddd254418a7bc7001052793c151c32b171e82af6c9b34df8cc403928dc9440"}
Oct 30 20:02:31.459813 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.459614    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp" event={"ID":"53821e33-7446-47cd-afd1-7489845cda73","Type":"ContainerDied","Data":"90148cccd46d391ab8dcbc8872ba80016a528ddb91c3d668ca160040441ecf0c"}
Oct 30 20:02:31.459813 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.459643    5704 scope.go:122] "RemoveContainer" containerID="70ddd254418a7bc7001052793c151c32b171e82af6c9b34df8cc403928dc9440"
Oct 30 20:02:31.461051 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.461009    5704 generic.go:356] "Generic (PLEG): container finished" podID="7d4049f4-af8d-407e-b344-b8730a24b4e2" containerID="3817b8a5739f04748945a8c737ab43ef2a715e3c67401e8a5463739e5581fa5d" exitCode=0
Oct 30 20:02:31.461163 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.461063    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt" event={"ID":"7d4049f4-af8d-407e-b344-b8730a24b4e2","Type":"ContainerDied","Data":"3817b8a5739f04748945a8c737ab43ef2a715e3c67401e8a5463739e5581fa5d"}
Oct 30 20:02:31.461163 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.461081    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"
Oct 30 20:02:31.461163 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.461109    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt" event={"ID":"7d4049f4-af8d-407e-b344-b8730a24b4e2","Type":"ContainerDied","Data":"5d9b93f2f243ee752b91610e2c20f70cf81f4e44230a91601262a85befad379b"}
Oct 30 20:02:31.468955 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.468813    5704 scope.go:122] "RemoveContainer" containerID="70ddd254418a7bc7001052793c151c32b171e82af6c9b34df8cc403928dc9440"
Oct 30 20:02:31.469113 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:02:31.469095    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70ddd254418a7bc7001052793c151c32b171e82af6c9b34df8cc403928dc9440\": container with ID starting with 70ddd254418a7bc7001052793c151c32b171e82af6c9b34df8cc403928dc9440 not found: ID does not exist" containerID="70ddd254418a7bc7001052793c151c32b171e82af6c9b34df8cc403928dc9440"
Oct 30 20:02:31.469162 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.469121    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70ddd254418a7bc7001052793c151c32b171e82af6c9b34df8cc403928dc9440"} err="failed to get container status \"70ddd254418a7bc7001052793c151c32b171e82af6c9b34df8cc403928dc9440\": rpc error: code = NotFound desc = could not find container \"70ddd254418a7bc7001052793c151c32b171e82af6c9b34df8cc403928dc9440\": container with ID starting with 70ddd254418a7bc7001052793c151c32b171e82af6c9b34df8cc403928dc9440 not found: ID does not exist"
Oct 30 20:02:31.469162 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.469138    5704 scope.go:122] "RemoveContainer" containerID="3817b8a5739f04748945a8c737ab43ef2a715e3c67401e8a5463739e5581fa5d"
Oct 30 20:02:31.476917 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.476901    5704 scope.go:122] "RemoveContainer" containerID="3817b8a5739f04748945a8c737ab43ef2a715e3c67401e8a5463739e5581fa5d"
Oct 30 20:02:31.477157 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:02:31.477141    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3817b8a5739f04748945a8c737ab43ef2a715e3c67401e8a5463739e5581fa5d\": container with ID starting with 3817b8a5739f04748945a8c737ab43ef2a715e3c67401e8a5463739e5581fa5d not found: ID does not exist" containerID="3817b8a5739f04748945a8c737ab43ef2a715e3c67401e8a5463739e5581fa5d"
Oct 30 20:02:31.477234 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.477162    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3817b8a5739f04748945a8c737ab43ef2a715e3c67401e8a5463739e5581fa5d"} err="failed to get container status \"3817b8a5739f04748945a8c737ab43ef2a715e3c67401e8a5463739e5581fa5d\": rpc error: code = NotFound desc = could not find container \"3817b8a5739f04748945a8c737ab43ef2a715e3c67401e8a5463739e5581fa5d\": container with ID starting with 3817b8a5739f04748945a8c737ab43ef2a715e3c67401e8a5463739e5581fa5d not found: ID does not exist"
Oct 30 20:02:31.612369 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.612346    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"]
Oct 30 20:02:31.616512 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.616492    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-extmod-b-f77d05-75dbd9cc88-cbkwt"]
Oct 30 20:02:31.718965 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.718904    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"]
Oct 30 20:02:31.725471 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:31.725451    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-extmod-b-f77d05-66cd886895-hlnrp"]
Oct 30 20:02:33.178183 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:33.178154    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="53821e33-7446-47cd-afd1-7489845cda73" path="/var/lib/kubelet/pods/53821e33-7446-47cd-afd1-7489845cda73/volumes"
Oct 30 20:02:33.178566 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:33.178505    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d4049f4-af8d-407e-b344-b8730a24b4e2" path="/var/lib/kubelet/pods/7d4049f4-af8d-407e-b344-b8730a24b4e2/volumes"
Oct 30 20:02:33.382788 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:02:33.382753    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 20:02:34.482558 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:34.482528    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:36.034015 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:36.033982    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:36.225197 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:36.225162    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:43.649696 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:43.649639    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" podUID="8ec41f19-292e-4376-a629-31951649bfe9" containerName="main" containerID="cri-o://1d4ed510ea862c0036122e42b0f49ab0ab25d41e9fa292977a58361252825259" gracePeriod=45
Oct 30 20:02:43.769372 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:43.769340    5704 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" podUID="8ec41f19-292e-4376-a629-31951649bfe9" containerName="main" probeResult="failure" output="Get \"https://10.134.0.62:8000/ready\": dial tcp 10.134.0.62:8000: connect: connection refused"
Oct 30 20:02:43.899151 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:43.899125    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 20:02:44.011103 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.011075    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8ec41f19-292e-4376-a629-31951649bfe9-kube-api-access-pb7bh\" (UniqueName: \"kubernetes.io/projected/8ec41f19-292e-4376-a629-31951649bfe9-kube-api-access-pb7bh\") pod \"8ec41f19-292e-4376-a629-31951649bfe9\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") "
Oct 30 20:02:44.011261 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.011137    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/8ec41f19-292e-4376-a629-31951649bfe9-tls-certs\" (UniqueName: \"kubernetes.io/secret/8ec41f19-292e-4376-a629-31951649bfe9-tls-certs\") pod \"8ec41f19-292e-4376-a629-31951649bfe9\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") "
Oct 30 20:02:44.011261 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.011165    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-tmp-dir\") pod \"8ec41f19-292e-4376-a629-31951649bfe9\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") "
Oct 30 20:02:44.011261 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.011207    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-home\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-home\") pod \"8ec41f19-292e-4376-a629-31951649bfe9\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") "
Oct 30 20:02:44.011261 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.011224    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-model-cache\") pod \"8ec41f19-292e-4376-a629-31951649bfe9\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") "
Oct 30 20:02:44.011456 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.011268    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-dshm\") pod \"8ec41f19-292e-4376-a629-31951649bfe9\" (UID: \"8ec41f19-292e-4376-a629-31951649bfe9\") "
Oct 30 20:02:44.011518 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.011468    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-tmp-dir" pod "8ec41f19-292e-4376-a629-31951649bfe9" (UID: "8ec41f19-292e-4376-a629-31951649bfe9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:44.011518 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.011483    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-home" pod "8ec41f19-292e-4376-a629-31951649bfe9" (UID: "8ec41f19-292e-4376-a629-31951649bfe9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:44.011518 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.011506    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-model-cache" pod "8ec41f19-292e-4376-a629-31951649bfe9" (UID: "8ec41f19-292e-4376-a629-31951649bfe9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:44.011685 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.011619    5704 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-tmp-dir\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:44.011685 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.011633    5704 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-home\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:44.011685 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.011641    5704 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-model-cache\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:44.013866 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.013838    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ec41f19-292e-4376-a629-31951649bfe9-tls-certs" pod "8ec41f19-292e-4376-a629-31951649bfe9" (UID: "8ec41f19-292e-4376-a629-31951649bfe9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:02:44.014011 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.013861    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-dshm" pod "8ec41f19-292e-4376-a629-31951649bfe9" (UID: "8ec41f19-292e-4376-a629-31951649bfe9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:02:44.014011 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.013962    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ec41f19-292e-4376-a629-31951649bfe9-kube-api-access-pb7bh" pod "8ec41f19-292e-4376-a629-31951649bfe9" (UID: "8ec41f19-292e-4376-a629-31951649bfe9"). InnerVolumeSpecName "kube-api-access-pb7bh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:02:44.112628 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.112598    5704 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ec41f19-292e-4376-a629-31951649bfe9-dshm\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:44.112628 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.112624    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pb7bh\" (UniqueName: \"kubernetes.io/projected/8ec41f19-292e-4376-a629-31951649bfe9-kube-api-access-pb7bh\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:44.112628 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.112633    5704 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ec41f19-292e-4376-a629-31951649bfe9-tls-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:44.532473 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.532443    5704 generic.go:356] "Generic (PLEG): container finished" podID="8ec41f19-292e-4376-a629-31951649bfe9" containerID="1d4ed510ea862c0036122e42b0f49ab0ab25d41e9fa292977a58361252825259" exitCode=0
Oct 30 20:02:44.532669 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.532508    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"
Oct 30 20:02:44.532669 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.532520    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" event={"ID":"8ec41f19-292e-4376-a629-31951649bfe9","Type":"ContainerDied","Data":"1d4ed510ea862c0036122e42b0f49ab0ab25d41e9fa292977a58361252825259"}
Oct 30 20:02:44.532669 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.532548    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp" event={"ID":"8ec41f19-292e-4376-a629-31951649bfe9","Type":"ContainerDied","Data":"ebcd1f18489d148ee86bb4862d9141b6f30657a7996723bf11d4bf3d1c11d802"}
Oct 30 20:02:44.532669 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.532565    5704 scope.go:122] "RemoveContainer" containerID="1d4ed510ea862c0036122e42b0f49ab0ab25d41e9fa292977a58361252825259"
Oct 30 20:02:44.544391 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.544368    5704 scope.go:122] "RemoveContainer" containerID="1d4ed510ea862c0036122e42b0f49ab0ab25d41e9fa292977a58361252825259"
Oct 30 20:02:44.544666 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:02:44.544646    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d4ed510ea862c0036122e42b0f49ab0ab25d41e9fa292977a58361252825259\": container with ID starting with 1d4ed510ea862c0036122e42b0f49ab0ab25d41e9fa292977a58361252825259 not found: ID does not exist" containerID="1d4ed510ea862c0036122e42b0f49ab0ab25d41e9fa292977a58361252825259"
Oct 30 20:02:44.544719 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.544678    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d4ed510ea862c0036122e42b0f49ab0ab25d41e9fa292977a58361252825259"} err="failed to get container status \"1d4ed510ea862c0036122e42b0f49ab0ab25d41e9fa292977a58361252825259\": rpc error: code = NotFound desc = could not find container \"1d4ed510ea862c0036122e42b0f49ab0ab25d41e9fa292977a58361252825259\": container with ID starting with 1d4ed510ea862c0036122e42b0f49ab0ab25d41e9fa292977a58361252825259 not found: ID does not exist"
Oct 30 20:02:44.690109 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.690081    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"]
Oct 30 20:02:44.736153 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:44.736128    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-1323a7/distinct-2-w4-1323a7-kserve-77bc55c65-4lmwp"]
Oct 30 20:02:45.178199 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:45.178166    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8ec41f19-292e-4376-a629-31951649bfe9" path="/var/lib/kubelet/pods/8ec41f19-292e-4376-a629-31951649bfe9/volumes"
Oct 30 20:02:53.875767 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:53.875734    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"]
Oct 30 20:02:53.876306 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:53.875916    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm" podUID="85b02ffa-cb78-4aa1-8b00-058192c2cb0d" containerName="maas-api" containerID="cri-o://94d5f96b48283bad5e8ff9e74266eb55b5a624bfef024ddf4bbbbfc0d83dcafc" gracePeriod=30
Oct 30 20:02:54.011988 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.011952    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:54.143739 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.143677    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 20:02:54.196748 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.196717    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-maas-api-tls\") pod \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") "
Oct 30 20:02:54.196897 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.196785    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-kube-api-access-lwvfb\" (UniqueName: \"kubernetes.io/projected/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-kube-api-access-lwvfb\") pod \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") "
Oct 30 20:02:54.196897 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.196814    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-metrics-certs\" (UniqueName: \"kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-metrics-certs\") pod \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") "
Oct 30 20:02:54.196989 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.196899    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-openshift-service-ca\") pod \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\" (UID: \"85b02ffa-cb78-4aa1-8b00-058192c2cb0d\") "
Oct 30 20:02:54.197366 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.197332    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-openshift-service-ca" pod "85b02ffa-cb78-4aa1-8b00-058192c2cb0d" (UID: "85b02ffa-cb78-4aa1-8b00-058192c2cb0d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:02:54.198866 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.198840    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-maas-api-tls" pod "85b02ffa-cb78-4aa1-8b00-058192c2cb0d" (UID: "85b02ffa-cb78-4aa1-8b00-058192c2cb0d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:02:54.198978 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.198929    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-kube-api-access-lwvfb" pod "85b02ffa-cb78-4aa1-8b00-058192c2cb0d" (UID: "85b02ffa-cb78-4aa1-8b00-058192c2cb0d"). InnerVolumeSpecName "kube-api-access-lwvfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:02:54.198978 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.198965    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-metrics-certs" pod "85b02ffa-cb78-4aa1-8b00-058192c2cb0d" (UID: "85b02ffa-cb78-4aa1-8b00-058192c2cb0d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Oct 30 20:02:54.298309 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.298276    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwvfb\" (UniqueName: \"kubernetes.io/projected/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-kube-api-access-lwvfb\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:54.298309 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.298301    5704 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-metrics-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:54.298309 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.298312    5704 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-openshift-service-ca\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:54.298553 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.298321    5704 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/85b02ffa-cb78-4aa1-8b00-058192c2cb0d-maas-api-tls\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:02:54.582008 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.581978    5704 generic.go:356] "Generic (PLEG): container finished" podID="85b02ffa-cb78-4aa1-8b00-058192c2cb0d" containerID="94d5f96b48283bad5e8ff9e74266eb55b5a624bfef024ddf4bbbbfc0d83dcafc" exitCode=0
Oct 30 20:02:54.582210 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.582026    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"
Oct 30 20:02:54.582210 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.582069    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm" event={"ID":"85b02ffa-cb78-4aa1-8b00-058192c2cb0d","Type":"ContainerDied","Data":"94d5f96b48283bad5e8ff9e74266eb55b5a624bfef024ddf4bbbbfc0d83dcafc"}
Oct 30 20:02:54.582210 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.582122    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm" event={"ID":"85b02ffa-cb78-4aa1-8b00-058192c2cb0d","Type":"ContainerDied","Data":"70e3e55d2e2d4687e262d011eecb1352ee77215e6a1aec186962d5b1b9bbbc9e"}
Oct 30 20:02:54.582210 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.582143    5704 scope.go:122] "RemoveContainer" containerID="94d5f96b48283bad5e8ff9e74266eb55b5a624bfef024ddf4bbbbfc0d83dcafc"
Oct 30 20:02:54.591464 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.591448    5704 scope.go:122] "RemoveContainer" containerID="94d5f96b48283bad5e8ff9e74266eb55b5a624bfef024ddf4bbbbfc0d83dcafc"
Oct 30 20:02:54.591707 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:02:54.591687    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94d5f96b48283bad5e8ff9e74266eb55b5a624bfef024ddf4bbbbfc0d83dcafc\": container with ID starting with 94d5f96b48283bad5e8ff9e74266eb55b5a624bfef024ddf4bbbbfc0d83dcafc not found: ID does not exist" containerID="94d5f96b48283bad5e8ff9e74266eb55b5a624bfef024ddf4bbbbfc0d83dcafc"
Oct 30 20:02:54.591755 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.591715    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94d5f96b48283bad5e8ff9e74266eb55b5a624bfef024ddf4bbbbfc0d83dcafc"} err="failed to get container status \"94d5f96b48283bad5e8ff9e74266eb55b5a624bfef024ddf4bbbbfc0d83dcafc\": rpc error: code = NotFound desc = could not find container \"94d5f96b48283bad5e8ff9e74266eb55b5a624bfef024ddf4bbbbfc0d83dcafc\": container with ID starting with 94d5f96b48283bad5e8ff9e74266eb55b5a624bfef024ddf4bbbbfc0d83dcafc not found: ID does not exist"
Oct 30 20:02:54.614879 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.614857    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 20:02:54.736053 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.736014    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"]
Oct 30 20:02:54.740473 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:54.740452    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-1323a7-586d7d798f-pr5pm"]
Oct 30 20:02:55.178688 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:55.178647    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85b02ffa-cb78-4aa1-8b00-058192c2cb0d" path="/var/lib/kubelet/pods/85b02ffa-cb78-4aa1-8b00-058192c2cb0d/volumes"
Oct 30 20:02:55.475473 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:55.475383    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:55.734693 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:55.734615    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:02:56.428885 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:02:56.428842    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:03.383003 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:03:03.382965    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 20:03:03.386225 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:03.386189    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:03.598163 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:03.598129    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:03.719361 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:03.719270    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"]
Oct 30 20:03:03.719541 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:03.719515    5704 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf" podUID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerName="istio-proxy" containerID="cri-o://ce8cba74dc4ebdbb15e5940855b59b1ebeaaa4cef4f24fe12ea660879a643d8e" gracePeriod=30
Oct 30 20:03:03.787330 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:03.787287    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:04.949321 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:04.949290    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:05.187454 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:05.187423    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:06.903565 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:06.903534    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:07.455215 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:07.455184    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-gpgrq_d000c687-d9f1-4c3b-ae0c-f4e6267306b9/limitador/0.log"
Oct 30 20:03:08.536162 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:08.536128    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5846b8599-xf797_9e0abe2b-6b0d-4c71-9c95-bf49784088f0/manager/0.log"
Oct 30 20:03:08.961899 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:08.961876    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 20:03:09.028074 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028026    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-socket\") pod \"a1b77c27-687a-477c-ab0a-35b33474ec49\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") "
Oct 30 20:03:09.028202 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028104    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-podinfo\") pod \"a1b77c27-687a-477c-ab0a-35b33474ec49\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") "
Oct 30 20:03:09.028202 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028134    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1b77c27-687a-477c-ab0a-35b33474ec49-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1b77c27-687a-477c-ab0a-35b33474ec49-istiod-ca-cert\") pod \"a1b77c27-687a-477c-ab0a-35b33474ec49\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") "
Oct 30 20:03:09.028202 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028165    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-token\" (UniqueName: \"kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-token\") pod \"a1b77c27-687a-477c-ab0a-35b33474ec49\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") "
Oct 30 20:03:09.028383 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028232    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-envoy\") pod \"a1b77c27-687a-477c-ab0a-35b33474ec49\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") "
Oct 30 20:03:09.028383 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028291    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-certs\") pod \"a1b77c27-687a-477c-ab0a-35b33474ec49\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") "
Oct 30 20:03:09.028383 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028310    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-socket" pod "a1b77c27-687a-477c-ab0a-35b33474ec49" (UID: "a1b77c27-687a-477c-ab0a-35b33474ec49"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:03:09.028383 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028327    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-data\") pod \"a1b77c27-687a-477c-ab0a-35b33474ec49\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") "
Oct 30 20:03:09.028383 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028361    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-kube-api-access-6vbpd\" (UniqueName: \"kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-kube-api-access-6vbpd\") pod \"a1b77c27-687a-477c-ab0a-35b33474ec49\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") "
Oct 30 20:03:09.028669 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028433    5704 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-credential-socket\") pod \"a1b77c27-687a-477c-ab0a-35b33474ec49\" (UID: \"a1b77c27-687a-477c-ab0a-35b33474ec49\") "
Oct 30 20:03:09.028669 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028580    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-certs" pod "a1b77c27-687a-477c-ab0a-35b33474ec49" (UID: "a1b77c27-687a-477c-ab0a-35b33474ec49"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:03:09.028669 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028618    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-data" pod "a1b77c27-687a-477c-ab0a-35b33474ec49" (UID: "a1b77c27-687a-477c-ab0a-35b33474ec49"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:03:09.028669 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028644    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1b77c27-687a-477c-ab0a-35b33474ec49-istiod-ca-cert" pod "a1b77c27-687a-477c-ab0a-35b33474ec49" (UID: "a1b77c27-687a-477c-ab0a-35b33474ec49"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Oct 30 20:03:09.028915 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028778    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-credential-socket" pod "a1b77c27-687a-477c-ab0a-35b33474ec49" (UID: "a1b77c27-687a-477c-ab0a-35b33474ec49"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:03:09.028915 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028852    5704 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-certs\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:09.028915 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028881    5704 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-data\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:09.028915 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028895    5704 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-workload-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:09.028915 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.028908    5704 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1b77c27-687a-477c-ab0a-35b33474ec49-istiod-ca-cert\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:09.030657 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.030623    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-token" pod "a1b77c27-687a-477c-ab0a-35b33474ec49" (UID: "a1b77c27-687a-477c-ab0a-35b33474ec49"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:03:09.030776 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.030752    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-podinfo" pod "a1b77c27-687a-477c-ab0a-35b33474ec49" (UID: "a1b77c27-687a-477c-ab0a-35b33474ec49"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Oct 30 20:03:09.030934 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.030911    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-kube-api-access-6vbpd" pod "a1b77c27-687a-477c-ab0a-35b33474ec49" (UID: "a1b77c27-687a-477c-ab0a-35b33474ec49"). InnerVolumeSpecName "kube-api-access-6vbpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Oct 30 20:03:09.031057 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.031013    5704 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-envoy" pod "a1b77c27-687a-477c-ab0a-35b33474ec49" (UID: "a1b77c27-687a-477c-ab0a-35b33474ec49"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Oct 30 20:03:09.130314 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.130254    5704 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-credential-socket\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:09.130314 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.130275    5704 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-podinfo\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:09.130314 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.130285    5704 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-token\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:09.130314 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.130293    5704 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1b77c27-687a-477c-ab0a-35b33474ec49-istio-envoy\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:09.130314 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.130302    5704 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vbpd\" (UniqueName: \"kubernetes.io/projected/a1b77c27-687a-477c-ab0a-35b33474ec49-kube-api-access-6vbpd\") on node \"ip-10-0-139-33.us-west-2.compute.internal\" DevicePath \"\""
Oct 30 20:03:09.216703 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.216675    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7c9b87d574-gpgrq"]
Oct 30 20:03:09.650709 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.650676    5704 generic.go:356] "Generic (PLEG): container finished" podID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerID="ce8cba74dc4ebdbb15e5940855b59b1ebeaaa4cef4f24fe12ea660879a643d8e" exitCode=0
Oct 30 20:03:09.651155 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.650723    5704 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"
Oct 30 20:03:09.651155 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.650763    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf" event={"ID":"a1b77c27-687a-477c-ab0a-35b33474ec49","Type":"ContainerDied","Data":"ce8cba74dc4ebdbb15e5940855b59b1ebeaaa4cef4f24fe12ea660879a643d8e"}
Oct 30 20:03:09.651155 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.650817    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf" event={"ID":"a1b77c27-687a-477c-ab0a-35b33474ec49","Type":"ContainerDied","Data":"e3a1be4e90b11286229ee34d506578678417df8f534c627c756ed2c038096681"}
Oct 30 20:03:09.651155 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.650840    5704 scope.go:122] "RemoveContainer" containerID="ce8cba74dc4ebdbb15e5940855b59b1ebeaaa4cef4f24fe12ea660879a643d8e"
Oct 30 20:03:09.659452 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.659434    5704 scope.go:122] "RemoveContainer" containerID="4e5a90ab0970efdd5afe369eed930a28d7ffe76bfdd2b42ec1cf83fd40a3b277"
Oct 30 20:03:09.666860 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.666843    5704 scope.go:122] "RemoveContainer" containerID="ce8cba74dc4ebdbb15e5940855b59b1ebeaaa4cef4f24fe12ea660879a643d8e"
Oct 30 20:03:09.667128 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:03:09.667109    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce8cba74dc4ebdbb15e5940855b59b1ebeaaa4cef4f24fe12ea660879a643d8e\": container with ID starting with ce8cba74dc4ebdbb15e5940855b59b1ebeaaa4cef4f24fe12ea660879a643d8e not found: ID does not exist" containerID="ce8cba74dc4ebdbb15e5940855b59b1ebeaaa4cef4f24fe12ea660879a643d8e"
Oct 30 20:03:09.667183 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.667135    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce8cba74dc4ebdbb15e5940855b59b1ebeaaa4cef4f24fe12ea660879a643d8e"} err="failed to get container status \"ce8cba74dc4ebdbb15e5940855b59b1ebeaaa4cef4f24fe12ea660879a643d8e\": rpc error: code = NotFound desc = could not find container \"ce8cba74dc4ebdbb15e5940855b59b1ebeaaa4cef4f24fe12ea660879a643d8e\": container with ID starting with ce8cba74dc4ebdbb15e5940855b59b1ebeaaa4cef4f24fe12ea660879a643d8e not found: ID does not exist"
Oct 30 20:03:09.667183 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.667151    5704 scope.go:122] "RemoveContainer" containerID="4e5a90ab0970efdd5afe369eed930a28d7ffe76bfdd2b42ec1cf83fd40a3b277"
Oct 30 20:03:09.667398 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:03:09.667381    5704 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e5a90ab0970efdd5afe369eed930a28d7ffe76bfdd2b42ec1cf83fd40a3b277\": container with ID starting with 4e5a90ab0970efdd5afe369eed930a28d7ffe76bfdd2b42ec1cf83fd40a3b277 not found: ID does not exist" containerID="4e5a90ab0970efdd5afe369eed930a28d7ffe76bfdd2b42ec1cf83fd40a3b277"
Oct 30 20:03:09.667451 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.667403    5704 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e5a90ab0970efdd5afe369eed930a28d7ffe76bfdd2b42ec1cf83fd40a3b277"} err="failed to get container status \"4e5a90ab0970efdd5afe369eed930a28d7ffe76bfdd2b42ec1cf83fd40a3b277\": rpc error: code = NotFound desc = could not find container \"4e5a90ab0970efdd5afe369eed930a28d7ffe76bfdd2b42ec1cf83fd40a3b277\": container with ID starting with 4e5a90ab0970efdd5afe369eed930a28d7ffe76bfdd2b42ec1cf83fd40a3b277 not found: ID does not exist"
Oct 30 20:03:09.804140 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.804105    5704 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"]
Oct 30 20:03:09.849859 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:09.849835    5704 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1323a7-openshift-default-7b64d489ff-jvmgf"]
Oct 30 20:03:10.253338 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:10.253312    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gmcx6_fca519ef-8ea0-4fdf-a9a2-e682e6896739/server/0.log"
Oct 30 20:03:10.778544 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:10.778515    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-k469z_8e6d648f-ac01-4171-afbb-7db22296c447/proxy/0.log"
Oct 30 20:03:11.178004 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:11.177929    5704 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1b77c27-687a-477c-ab0a-35b33474ec49" path="/var/lib/kubelet/pods/a1b77c27-687a-477c-ab0a-35b33474ec49/volumes"
Oct 30 20:03:12.794708 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:12.794666    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:19.944457 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:19.944414    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:22.602698 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:22.602667    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5846b8599-xf797_9e0abe2b-6b0d-4c71-9c95-bf49784088f0/manager/0.log"
Oct 30 20:03:24.396819 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:24.396786    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gmcx6_fca519ef-8ea0-4fdf-a9a2-e682e6896739/server/0.log"
Oct 30 20:03:24.897233 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:24.897206    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-k469z_8e6d648f-ac01-4171-afbb-7db22296c447/proxy/0.log"
Oct 30 20:03:26.029381 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:26.029350    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-97d69cf74-lxpgv_b19117a1-93c9-4a5a-a713-0f2a41bace8f/maas-api/0.log"
Oct 30 20:03:32.584960 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:32.584901    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-gpgrq_d000c687-d9f1-4c3b-ae0c-f4e6267306b9/limitador/0.log"
Oct 30 20:03:33.383285 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:03:33.383240    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 20:03:33.928210 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:33.928185    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-2tk6r_617868bc-e87c-48c3-b358-7210bf1f388c/discovery/0.log"
Oct 30 20:03:34.304428 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:34.304397    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6df7c6869c-kqhtm_72ab8598-6730-4763-8a86-4c5ba57c6b65/kube-auth-proxy/0.log"
Oct 30 20:03:37.129489 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:37.129455    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-7b998b79d9-fvvkh_244df3be-6afb-4788-a07e-533b1d3cc737/main/0.log"
Oct 30 20:03:37.689503 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:37.689469    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-69b588897d-dc9m6_7911229c-1ff9-4b92-bfcd-c060a9988f8e/main/0.log"
Oct 30 20:03:38.221399 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:38.221375    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_echo-simulator-kserve-64b6f8f96c-g4zrw_1ada7c08-cef2-46d5-96b6-a6a8fddbc84a/main/0.log"
Oct 30 20:03:38.772951 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:38.772914    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5f5c4b459f-sckv8_fc21e17b-89c9-4a14-ad31-21d3ae548ee2/main/0.log"
Oct 30 20:03:49.826052 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.825964    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:49.852920 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.852888    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8gmw7/must-gather-wbw55"]
Oct 30 20:03:49.853312 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853299    5704 cpu_manager.go:414] "containerMap: removing container" podUID="85b02ffa-cb78-4aa1-8b00-058192c2cb0d" containerName="maas-api"
Oct 30 20:03:49.853379 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853314    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85b02ffa-cb78-4aa1-8b00-058192c2cb0d" containerName="maas-api"
Oct 30 20:03:49.853379 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853330    5704 cpu_manager.go:414] "containerMap: removing container" podUID="53821e33-7446-47cd-afd1-7489845cda73" containerName="payload-pre-processing"
Oct 30 20:03:49.853379 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853336    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="53821e33-7446-47cd-afd1-7489845cda73" containerName="payload-pre-processing"
Oct 30 20:03:49.853379 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853352    5704 cpu_manager.go:414] "containerMap: removing container" podUID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerName="istio-proxy"
Oct 30 20:03:49.853379 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853357    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerName="istio-proxy"
Oct 30 20:03:49.853379 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853365    5704 cpu_manager.go:414] "containerMap: removing container" podUID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerName="istio-proxy"
Oct 30 20:03:49.853379 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853371    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerName="istio-proxy"
Oct 30 20:03:49.853379 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853376    5704 cpu_manager.go:414] "containerMap: removing container" podUID="8ec41f19-292e-4376-a629-31951649bfe9" containerName="main"
Oct 30 20:03:49.853379 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853381    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8ec41f19-292e-4376-a629-31951649bfe9" containerName="main"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853390    5704 cpu_manager.go:414] "containerMap: removing container" podUID="c56c7254-9b4c-47d9-b26c-1dd3384b04fd" containerName="payload-processing"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853394    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c56c7254-9b4c-47d9-b26c-1dd3384b04fd" containerName="payload-processing"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853408    5704 cpu_manager.go:414] "containerMap: removing container" podUID="edd8b872-d7f3-4cc3-b16c-59b29329b524" containerName="payload-processing"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853415    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="edd8b872-d7f3-4cc3-b16c-59b29329b524" containerName="payload-processing"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853427    5704 cpu_manager.go:414] "containerMap: removing container" podUID="6db8a91c-6926-4df9-8c74-453363556742" containerName="payload-processing"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853432    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6db8a91c-6926-4df9-8c74-453363556742" containerName="payload-processing"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853440    5704 cpu_manager.go:414] "containerMap: removing container" podUID="7d4049f4-af8d-407e-b344-b8730a24b4e2" containerName="payload-processing"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853445    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d4049f4-af8d-407e-b344-b8730a24b4e2" containerName="payload-processing"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853453    5704 cpu_manager.go:414] "containerMap: removing container" podUID="d4171f5d-d080-40d0-8a48-f592ba37f8d6" containerName="istio-proxy"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853457    5704 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d4171f5d-d080-40d0-8a48-f592ba37f8d6" containerName="istio-proxy"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853514    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="edd8b872-d7f3-4cc3-b16c-59b29329b524" containerName="payload-processing"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853521    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="6db8a91c-6926-4df9-8c74-453363556742" containerName="payload-processing"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853528    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d4049f4-af8d-407e-b344-b8730a24b4e2" containerName="payload-processing"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853534    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="d4171f5d-d080-40d0-8a48-f592ba37f8d6" containerName="istio-proxy"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853541    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="85b02ffa-cb78-4aa1-8b00-058192c2cb0d" containerName="maas-api"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853548    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="53821e33-7446-47cd-afd1-7489845cda73" containerName="payload-pre-processing"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853554    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerName="istio-proxy"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853561    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1b77c27-687a-477c-ab0a-35b33474ec49" containerName="istio-proxy"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853568    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="8ec41f19-292e-4376-a629-31951649bfe9" containerName="main"
Oct 30 20:03:49.853704 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.853573    5704 memory_manager.go:365] "RemoveStaleState removing state" podUID="c56c7254-9b4c-47d9-b26c-1dd3384b04fd" containerName="payload-processing"
Oct 30 20:03:49.856596 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.856578    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8gmw7/must-gather-wbw55"
Oct 30 20:03:49.878548 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.878521    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a5807e39-2be7-4be2-8192-91c407fad757-must-gather-output\") pod \"must-gather-wbw55\" (UID: \"a5807e39-2be7-4be2-8192-91c407fad757\") " pod="openshift-must-gather-8gmw7/must-gather-wbw55"
Oct 30 20:03:49.878655 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.878589    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99xq4\" (UniqueName: \"kubernetes.io/projected/a5807e39-2be7-4be2-8192-91c407fad757-kube-api-access-99xq4\") pod \"must-gather-wbw55\" (UID: \"a5807e39-2be7-4be2-8192-91c407fad757\") " pod="openshift-must-gather-8gmw7/must-gather-wbw55"
Oct 30 20:03:49.918134 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.918111    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8gmw7/must-gather-wbw55"]
Oct 30 20:03:49.951981 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.951933    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8gmw7\"/\"kube-root-ca.crt\""
Oct 30 20:03:49.952192 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.952117    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8gmw7\"/\"default-dockercfg-g4b9c\""
Oct 30 20:03:49.952192 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.952160    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8gmw7\"/\"openshift-service-ca.crt\""
Oct 30 20:03:49.979466 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.979441    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99xq4\" (UniqueName: \"kubernetes.io/projected/a5807e39-2be7-4be2-8192-91c407fad757-kube-api-access-99xq4\") pod \"must-gather-wbw55\" (UID: \"a5807e39-2be7-4be2-8192-91c407fad757\") " pod="openshift-must-gather-8gmw7/must-gather-wbw55"
Oct 30 20:03:49.979616 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.979525    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a5807e39-2be7-4be2-8192-91c407fad757-must-gather-output\") pod \"must-gather-wbw55\" (UID: \"a5807e39-2be7-4be2-8192-91c407fad757\") " pod="openshift-must-gather-8gmw7/must-gather-wbw55"
Oct 30 20:03:49.979878 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:49.979859    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/a5807e39-2be7-4be2-8192-91c407fad757-must-gather-output\") pod \"must-gather-wbw55\" (UID: \"a5807e39-2be7-4be2-8192-91c407fad757\") " pod="openshift-must-gather-8gmw7/must-gather-wbw55"
Oct 30 20:03:50.072979 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:50.072945    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99xq4\" (UniqueName: \"kubernetes.io/projected/a5807e39-2be7-4be2-8192-91c407fad757-kube-api-access-99xq4\") pod \"must-gather-wbw55\" (UID: \"a5807e39-2be7-4be2-8192-91c407fad757\") " pod="openshift-must-gather-8gmw7/must-gather-wbw55"
Oct 30 20:03:50.170690 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:50.170614    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8gmw7/must-gather-wbw55"
Oct 30 20:03:50.178508 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:50.178479    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:03:50.561967 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:50.561935    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8gmw7/must-gather-wbw55"]
Oct 30 20:03:50.565147 ip-10-0-139-33 kubenswrapper[5704]: W1030 20:03:50.565114    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda5807e39_2be7_4be2_8192_91c407fad757.slice/crio-cb7b63786b82438cfe7127f37279d52ef5bd03d35e5c25b0ea35f11f4f873b1e WatchSource:0}: Error finding container cb7b63786b82438cfe7127f37279d52ef5bd03d35e5c25b0ea35f11f4f873b1e: Status 404 returned error can't find the container with id cb7b63786b82438cfe7127f37279d52ef5bd03d35e5c25b0ea35f11f4f873b1e
Oct 30 20:03:50.566952 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:50.566938    5704 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Oct 30 20:03:50.841596 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:50.841517    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8gmw7/must-gather-wbw55" event={"ID":"a5807e39-2be7-4be2-8192-91c407fad757","Type":"ContainerStarted","Data":"cb7b63786b82438cfe7127f37279d52ef5bd03d35e5c25b0ea35f11f4f873b1e"}
Oct 30 20:03:59.714523 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:03:59.714486    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-wpm6z_b67fee5e-0972-4cfc-987d-b0ae79e6589d/global-pull-secret-syncer/0.log"
Oct 30 20:04:00.897665 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:00.897632    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8gmw7/must-gather-wbw55" event={"ID":"a5807e39-2be7-4be2-8192-91c407fad757","Type":"ContainerStarted","Data":"59478a1cd8bdd12658abe7c6960f7e85b1e5e9c6d25432162b38f1a0369a5470"}
Oct 30 20:04:00.898093 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:00.897672    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8gmw7/must-gather-wbw55" event={"ID":"a5807e39-2be7-4be2-8192-91c407fad757","Type":"ContainerStarted","Data":"6d11cec8c595233c69ae756beaf0be186e6b44dfa63a59e3a32b47121ec4897e"}
Oct 30 20:04:01.009071 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:01.009001    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-8gmw7/must-gather-wbw55" podStartSLOduration=2.317685873 podStartE2EDuration="12.008988317s" totalImagesPullingTime="9.691302444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 20:03:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:01.007548974 +0000 UTC m=+4613.953716878" watchObservedRunningTime="2026-10-30 20:04:01.008988317 +0000 UTC m=+4613.955156251"
Oct 30 20:04:01.660190 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:01.660158    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log"
Oct 30 20:04:03.381969 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:04:03.381934    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 20:04:03.523912 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:03.523885    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-33.us-west-2.compute.internal_9a61da0484b7ed0271380ee2068515a4/haproxy/0.log"
Oct 30 20:04:04.496244 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:04.496214    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5846b8599-xf797_9e0abe2b-6b0d-4c71-9c95-bf49784088f0/manager/0.log"
Oct 30 20:04:04.998057 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:04.998009    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5846b8599-xf797_9e0abe2b-6b0d-4c71-9c95-bf49784088f0/manager/0.log"
Oct 30 20:04:06.078804 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:06.078772    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gmcx6_fca519ef-8ea0-4fdf-a9a2-e682e6896739/server/0.log"
Oct 30 20:04:06.677665 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:06.677636    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gmcx6_fca519ef-8ea0-4fdf-a9a2-e682e6896739/server/0.log"
Oct 30 20:04:07.521297 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:07.521266    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-k469z_8e6d648f-ac01-4171-afbb-7db22296c447/proxy/0.log"
Oct 30 20:04:07.816749 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:07.816669    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:07.979843 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:07.979813    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-k469z_8e6d648f-ac01-4171-afbb-7db22296c447/proxy/0.log"
Oct 30 20:04:09.278302 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.278273    5704 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"]
Oct 30 20:04:09.284684 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.284662    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.338096 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.338071    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j68lw\"/\"openshift-service-ca.crt\""
Oct 30 20:04:09.338788 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.338771    5704 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-j68lw\"/\"kube-root-ca.crt\""
Oct 30 20:04:09.362608 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.362579    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zn5l\" (UniqueName: \"kubernetes.io/projected/6a02ba04-4176-47e8-abb6-ff74c874c727-kube-api-access-9zn5l\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.362608 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.362607    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a02ba04-4176-47e8-abb6-ff74c874c727-sys\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.362759 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.362629    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6a02ba04-4176-47e8-abb6-ff74c874c727-podres\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.362759 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.362739    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6a02ba04-4176-47e8-abb6-ff74c874c727-proc\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.362828 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.362758    5704 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a02ba04-4176-47e8-abb6-ff74c874c727-lib-modules\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.384202 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.384175    5704 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-j68lw\"/\"default-dockercfg-plqkf\""
Oct 30 20:04:09.385545 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.385519    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"]
Oct 30 20:04:09.464298 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.464253    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zn5l\" (UniqueName: \"kubernetes.io/projected/6a02ba04-4176-47e8-abb6-ff74c874c727-kube-api-access-9zn5l\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.464298 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.464298    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a02ba04-4176-47e8-abb6-ff74c874c727-sys\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.464544 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.464317    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6a02ba04-4176-47e8-abb6-ff74c874c727-podres\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.464544 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.464367    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6a02ba04-4176-47e8-abb6-ff74c874c727-proc\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.464544 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.464384    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6a02ba04-4176-47e8-abb6-ff74c874c727-sys\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.464544 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.464393    5704 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a02ba04-4176-47e8-abb6-ff74c874c727-lib-modules\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.464544 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.464463    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6a02ba04-4176-47e8-abb6-ff74c874c727-lib-modules\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.464544 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.464463    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6a02ba04-4176-47e8-abb6-ff74c874c727-podres\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.464544 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.464493    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6a02ba04-4176-47e8-abb6-ff74c874c727-proc\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.523294 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.523263    5704 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zn5l\" (UniqueName: \"kubernetes.io/projected/6a02ba04-4176-47e8-abb6-ff74c874c727-kube-api-access-9zn5l\") pod \"perf-node-gather-daemonset-g4hg2\" (UID: \"6a02ba04-4176-47e8-abb6-ff74c874c727\") " pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.598685 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.598611    5704 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:09.975520 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:09.975494    5704 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"]
Oct 30 20:04:09.977979 ip-10-0-139-33 kubenswrapper[5704]: W1030 20:04:09.977949    5704 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6a02ba04_4176_47e8_abb6_ff74c874c727.slice/crio-9b1893273c0356dd6e2ed345898d0be5db99f728c5c6871176f8a5e0859abff6 WatchSource:0}: Error finding container 9b1893273c0356dd6e2ed345898d0be5db99f728c5c6871176f8a5e0859abff6: Status 404 returned error can't find the container with id 9b1893273c0356dd6e2ed345898d0be5db99f728c5c6871176f8a5e0859abff6
Oct 30 20:04:10.417285 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:10.417257    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5846b8599-xf797_9e0abe2b-6b0d-4c71-9c95-bf49784088f0/manager/0.log"
Oct 30 20:04:10.943534 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:10.943498    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2" event={"ID":"6a02ba04-4176-47e8-abb6-ff74c874c727","Type":"ContainerStarted","Data":"4ab9c78aa92161f99f237ebe43e40ee5bd1f7f97fd3289d09390a300c7337156"}
Oct 30 20:04:10.943719 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:10.943562    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:10.943719 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:10.943582    5704 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2" event={"ID":"6a02ba04-4176-47e8-abb6-ff74c874c727","Type":"ContainerStarted","Data":"9b1893273c0356dd6e2ed345898d0be5db99f728c5c6871176f8a5e0859abff6"}
Oct 30 20:04:11.025686 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:11.025658    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5846b8599-xf797_9e0abe2b-6b0d-4c71-9c95-bf49784088f0/manager/0.log"
Oct 30 20:04:11.056253 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:11.056209    5704 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2" podStartSLOduration=2.056195741 podStartE2EDuration="2.056195741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 20:04:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 20:04:11.053771822 +0000 UTC m=+4623.999939725" watchObservedRunningTime="2026-10-30 20:04:11.056195741 +0000 UTC m=+4624.002363645"
Oct 30 20:04:11.097923 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:11.097902    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-gpgrq_d000c687-d9f1-4c3b-ae0c-f4e6267306b9/limitador/0.log"
Oct 30 20:04:12.273894 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:12.273860    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gmcx6_fca519ef-8ea0-4fdf-a9a2-e682e6896739/server/0.log"
Oct 30 20:04:12.839190 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:12.839161    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gmcx6_fca519ef-8ea0-4fdf-a9a2-e682e6896739/server/0.log"
Oct 30 20:04:13.461292 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:13.461262    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-k469z_8e6d648f-ac01-4171-afbb-7db22296c447/proxy/0.log"
Oct 30 20:04:13.959974 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:13.959940    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-k469z_8e6d648f-ac01-4171-afbb-7db22296c447/proxy/0.log"
Oct 30 20:04:14.551069 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:14.551025    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fffcbe9-25c2-4796-812a-f7ac6278ceab/alertmanager/0.log"
Oct 30 20:04:14.955523 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:14.955413    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fffcbe9-25c2-4796-812a-f7ac6278ceab/config-reloader/0.log"
Oct 30 20:04:15.361220 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:15.361181    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fffcbe9-25c2-4796-812a-f7ac6278ceab/kube-rbac-proxy-web/0.log"
Oct 30 20:04:15.765862 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:15.765833    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fffcbe9-25c2-4796-812a-f7ac6278ceab/kube-rbac-proxy/0.log"
Oct 30 20:04:16.170428 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:16.170357    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fffcbe9-25c2-4796-812a-f7ac6278ceab/kube-rbac-proxy-metric/0.log"
Oct 30 20:04:16.246809 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:16.246784    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5846b8599-xf797_9e0abe2b-6b0d-4c71-9c95-bf49784088f0/manager/0.log"
Oct 30 20:04:16.595384 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:16.595354    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fffcbe9-25c2-4796-812a-f7ac6278ceab/prom-label-proxy/0.log"
Oct 30 20:04:16.848087 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:16.847988    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-5846b8599-xf797_9e0abe2b-6b0d-4c71-9c95-bf49784088f0/manager/0.log"
Oct 30 20:04:16.958701 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:16.958676    5704 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-j68lw/perf-node-gather-daemonset-g4hg2"
Oct 30 20:04:17.092211 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:17.092184    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_2fffcbe9-25c2-4796-812a-f7ac6278ceab/init-config-reloader/0.log"
Oct 30 20:04:17.840058 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:17.840020    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gmcx6_fca519ef-8ea0-4fdf-a9a2-e682e6896739/server/0.log"
Oct 30 20:04:18.014802 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:18.014769    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-j6hsl_0e786874-ce56-46f8-87f5-920dacacd9fc/kube-state-metrics/0.log"
Oct 30 20:04:18.413490 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:18.413455    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-j6hsl_0e786874-ce56-46f8-87f5-920dacacd9fc/kube-rbac-proxy-main/0.log"
Oct 30 20:04:18.484159 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:18.484134    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-gmcx6_fca519ef-8ea0-4fdf-a9a2-e682e6896739/server/0.log"
Oct 30 20:04:18.822003 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:18.821969    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-5dc8d7d89b-j6hsl_0e786874-ce56-46f8-87f5-920dacacd9fc/kube-rbac-proxy-self/0.log"
Oct 30 20:04:18.833330 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:18.833310    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-k469z_8e6d648f-ac01-4171-afbb-7db22296c447/proxy/0.log"
Oct 30 20:04:19.252755 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:19.252728    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-85d7bcfb5c-6vmfw_263a55c9-0e5a-47bc-bec1-20cd01e12942/metrics-server/0.log"
Oct 30 20:04:19.510087 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:19.510015    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-9c5d49bcd-k469z_8e6d648f-ac01-4171-afbb-7db22296c447/proxy/0.log"
Oct 30 20:04:19.672618 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:19.672587    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-77f4f76594-fnkq7_3acfed5c-f071-4450-9e72-d923cc492d3f/monitoring-plugin/0.log"
Oct 30 20:04:20.295300 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:20.295259    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:21.346133 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:21.346103    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qdr6m_ca586ba2-2d05-45f8-aab9-127b93ea1c60/node-exporter/0.log"
Oct 30 20:04:21.749028 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:21.748997    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qdr6m_ca586ba2-2d05-45f8-aab9-127b93ea1c60/kube-rbac-proxy/0.log"
Oct 30 20:04:22.153394 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:22.153319    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qdr6m_ca586ba2-2d05-45f8-aab9-127b93ea1c60/init-textfile/0.log"
Oct 30 20:04:23.423903 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:23.423872    5704 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-nv7ql_a6c94923-1962-425d-8e53-ad523e78619b/konnectivity-agent/0.log" limit=1024
Oct 30 20:04:23.725060 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:23.724974    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-2tk6r_617868bc-e87c-48c3-b358-7210bf1f388c/discovery/0.log"
Oct 30 20:04:24.591197 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:24.591166    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6df7c6869c-kqhtm_72ab8598-6730-4763-8a86-4c5ba57c6b65/kube-auth-proxy/0.log"
Oct 30 20:04:24.628816 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:24.628785    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-2tk6r_617868bc-e87c-48c3-b358-7210bf1f388c/discovery/0.log"
Oct 30 20:04:24.908064 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:24.907975    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6df7c6869c-kqhtm_72ab8598-6730-4763-8a86-4c5ba57c6b65/kube-auth-proxy/0.log"
Oct 30 20:04:25.638719 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:25.638689    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-5v6xc_42b2d2d6-78f0-457a-b182-09dd65dc3372/kube-rbac-proxy-main/0.log"
Oct 30 20:04:26.124441 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:26.124409    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-5v6xc_42b2d2d6-78f0-457a-b182-09dd65dc3372/kube-rbac-proxy-self/0.log"
Oct 30 20:04:26.531607 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:26.531576    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-7bd7489957-5v6xc_42b2d2d6-78f0-457a-b182-09dd65dc3372/openshift-state-metrics/0.log"
Oct 30 20:04:31.536955 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:31.536928    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76b58b9cbb-gctk8_287c6859-11cf-4a66-a3f8-b866c5477bb1/telemeter-client/0.log"
Oct 30 20:04:31.975672 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:31.975606    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76b58b9cbb-gctk8_287c6859-11cf-4a66-a3f8-b866c5477bb1/reload/0.log"
Oct 30 20:04:32.527450 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:32.527420    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76b58b9cbb-gctk8_287c6859-11cf-4a66-a3f8-b866c5477bb1/kube-rbac-proxy/0.log"
Oct 30 20:04:33.280426 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:33.280388    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-2tk6r_617868bc-e87c-48c3-b358-7210bf1f388c/discovery/0.log"
Oct 30 20:04:33.384358 ip-10-0-139-33 kubenswrapper[5704]: E1030 20:04:33.384322    5704 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-6d2f8" containerName="extractor"
Oct 30 20:04:33.435858 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:33.435819    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d769f5588-7flhp_9ae72d3a-8b68-4d30-93d7-f03418559928/thanos-query/0.log"
Oct 30 20:04:33.564880 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:33.564793    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6df7c6869c-kqhtm_72ab8598-6730-4763-8a86-4c5ba57c6b65/kube-auth-proxy/0.log"
Oct 30 20:04:33.853729 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:33.853647    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d769f5588-7flhp_9ae72d3a-8b68-4d30-93d7-f03418559928/kube-rbac-proxy-web/0.log"
Oct 30 20:04:33.895385 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:33.895356    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-gpgrq_d000c687-d9f1-4c3b-ae0c-f4e6267306b9/limitador/0.log"
Oct 30 20:04:34.225903 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:34.225828    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-7c9b87d574-gpgrq_d000c687-d9f1-4c3b-ae0c-f4e6267306b9/limitador/0.log"
Oct 30 20:04:34.263700 ip-10-0-139-33 kubenswrapper[5704]: I1030 20:04:34.263674    5704 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-5d769f5588-7flhp_9ae72d3a-8b68-4d30-93d7-f03418559928/kube-rbac-proxy/0.log"